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Preface 



The Department of Homeland Security (DHS) Office of Inspector General (OIG) was established by 
the Homeland Security Act of 2002 (Public Law 107-296) by amendment to the Inspector General 
Act of 1978. This is one of a series of audit, inspection, and special reports prepared by our office as 
part of our DHS oversight responsibility to promote economy, effectiveness, and efficiency within 
the department. 

This report provides an assessment of the Federal Emergency Management Agency's (FEMA) 
Multi-Hazard Flood Map Modernization Program. It addresses FEMA's program management 
approach, coordination with stakeholders, and acquisition as well as use of information technology 
(IT) to meet map modernization goals. The report is based on interviews with DHS and other 
federal, state, local, and contracting officials nationwide; direct observations; and, a review of 
applicable documents. 

The recommendations herein have been developed to the best knowledge available to our office, and 
have been discussed in draft with those responsible for implementation. It is our hope that this 
report will result in more effective, efficient, and economical operations. We express our 
appreciation to all of those who contributed to the preparation of this report. 




Richard L. Skinner 
Inspector General 
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Executive Summary 

Floods are among the most frequent and costly of all natural disasters and 
have great impact in terms of economic and human losses each year. Property 
damage resulting from floods totals over $5 billion annually and caused more 
than 900 deaths from FY 1992 to 2001. Communities with effective and 
accurate flood maps are prepared to prevent not only economic devastation, 
but to preserve life as well. Since 1978, FEMA has been charged with 
assisting communities by producing flood maps that detail areas at risk; 
identify where flood insurance is needed; and, help limit construction within 
flood zones. However, the majority of FEMA's maps are outdated and in 
unalterable paper format. In response to demands for more accurate mapping 
products, FEMA has embarked on a six-year, $1,475 billion program to 
update and digitize the nation's flood maps. 

As part of our ongoing responsibility to assess the efficiency and effectiveness 
of departmental programs and operations, we conducted an audit of FEMA's 
Multi-Hazard Flood Map Modernization Program. The objectives of our audit 
were to assess FEMA's management approach; coordination with federal, 
state, and local entities; and, acquisition and use of technology to meet map 
modernization program objectives. The scope and methodology of this audit 
are discussed in Appendix A. 

As a result of our audit, we determined that while FEMA is making progress 
in map modernization, a number of significant challenges remain. 
Specifically, FEMA has developed a plan that outlines the priorities, 
resources, and standards for accomplishing map modernization in 
communities across the U.S. However, because of budget limitations, 
FEMA's plan does not reflect user or funding needs. Also, the plan does not 
provide guidance on how new mapping standards will be achieved. Due to 
these deficiencies, the plan discourages stakeholder buy-in and may not help 
FEMA meet its map modernization schedule and quality goals. A modified 
plan and improved FEMA control of map modernization costs, schedule, and 
performance would help ensure effective program results. 

Further, FEMA has enhanced its efforts to partner and communicate with its 
mapping stakeholders — but the agency has not maximized the benefits 
possible through these relationships. For instance, although FEMA has 
improved coordination of federal mapping activities, FEMA has not instituted 
the policies, agreements, or information sharing mechanisms to effectively 
support these interagency working arrangements. Similarly, while FEMA is 
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partnering more effectively with state and local government entities, these 
stakeholders are not fully engaged or aware of key map modernization 
objectives. A FEMA call center, established to address stakeholder inquiries, 
does not provide consistently effective service or accurate information, 
potentially placing communities at risk. Addressing these issues will help 
ensure effective use of intergovernmental resources, stakeholder commitment, 
and ultimate success in meeting overall map modernization goals. 

Additionally, as part of its map modernization efforts, FEMA is developing a 
web-based technology platform and tools to support efficient production and 
sharing of digital maps. However, FEMA's IT development approach has 
limited program progress: unclear contractor expectations; underestimation of 
program scope and complexity; and, poorly defined requirements have 
resulted in significant system acquisition delays and cost overruns. Taking 
steps to assess progress, identify shortcomings, and make adjustments will 
help FEMA ensure that the technology acquisition achieves the functionality 
and cost savings it anticipates. 

Background 

As a component of DHS' Emergency Preparedness and Response Directorate, 
FEMA's mission is to "reduce loss of life and property and protect our 
nation's critical infrastructure from all types of hazards through a 
comprehensive, risk-based, emergency management program of mitigation, 
preparedness, response and recovery." FEMA's Mitigation Division manages 
the National Flood Insurance Program, which was created in 1968 to reduce 
flood damage by identifying flood risks, encouraging sound floodplain 
management practices, and providing a mechanism through which citizens can 
insure their investments. To help determine insurance requirements, FEMA 
maintains an inventory of over 90,000 flood map panels. (See Figure 1.) 
However, 70 percent of the maps are more than 10 years old; many of the 
floodplains depicted on them were hand-drawn and are difficult to update. 
Continued community development over time has generally rendered the 
maps inaccurate and obsolete. 




Figure 1 : Example of Flood Map Panel 
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Pursuant to the National Flood Insurance Reform Act of 1994} Congress 
created the Technical Mapping Advisory Council. The Council was charged 
with evaluating the production, distribution, and use of flood insurance rate 
maps and making recommendations regarding cost-effective improvements in 
the accuracy, quality, and distribution of mapping products. The Council 
made recommendations on using standards and guidelines in the program, too. 
Membership of the Council consisted of a FEMA Director designee as well as 
ten representatives from federal and state governments, professional 
associations with mapping interests, and lending institutions. The Council's 
final report, issued in 2000, included four general themes on which the Multi- 
Hazard Flood Map Modernization Program ultimately would be based: 

• Providing additional resources for floodplain maps 

• Building interest and support for modernizing maps 

• Building partnerships to accomplish National Flood Insurance 
Program objectives 

• Creating a fully digital mapping environment 

The Council's recommendations, coupled with new federal direction on 
geospatial information sharing, led to a more partnership-oriented approach to 
flood mapping. For example, the Office of Management and Budget required 
that federal agencies work together, as well as with state and local 
governments and private industry, on geospatial data collection as a means of 
reducing duplicative cost and effort. Executive Order 12906, part of the 
President' s e-government initiative, further strengthened policies on 
information sharing outlined in Circular A- 16. 3 

In line with this direction, FEMA's initial 1997 map modernization plan 
included strategies for working with other government entities to update flood 
maps and streamline risk mitigation operations. The map modernization 
program has four key objectives: (1) establish and maintain a premier data 
collection and delivery system; (2) achieve effective program management; 
(3) build and maintain mutually beneficial partnerships; and, (4) expand and 
better inform the user community. By producing more accurate and 
accessible flood maps, FEMA expects to benefit communities that use the 
maps to establish zoning and building standards, property owners that access 
the maps on the internet to see if they need to obtain flood insurance, and 
government officials that reference the maps to accurately locate 
infrastructure and transportation systems to help manage homeland security 
risks. 



1 Public Law 103-325, Title V, Section 576. 

2 Coordination of Geographic Information and Related Spatial Data Activities, Office of Management and Budget 
Circular A-16, August 19, 2002. 

3 Coordinating Geographic Data Acquisition and Access: The National Spatial Data Infrastructure, Executive Order 
12906, April 11, 1994. 

Challenges in FEMA's Flood Map Modernization Program 



Page 3 



However, because flood insurance revenues alone historically funded map 
production, FEMA needed additional resources to move forward with its map 
modernization plan. Congress provided FEMA with moderate annual funding 
increases from FY 2000 to 2002 to support establishment of new standards 
and guidelines for developing accurate, digital flood maps, expanding 
mapping partnership efforts, and initiating best practices nationwide. With a 
budget of $1,475 billion for a six-year period beginning in FY 2003, FEMA 
has been able to move forward with its map modernization program. 

FEMA's OIG and the Government Accountability Office (GAO) reviewed 
and reported on initial efforts to modernize flood maps. In September 2000, 
the FEMA OIG reported that FEMA's methodology was sound, requirements 
were reasonable, and no major cost elements were overlooked. 4 However, the 
OIG recommended that FEMA improve the accuracy of its cost estimate by 
verifying that data used in the estimate were accurate and that planned 
mapping initiatives were cost effective. In addition, the OIG recommended 
that FEMA maintain historical cost data on flood studies and other mapping- 
related costs and refine its estimates of the impact of technological advances 
on mapping costs. 

Further, in a March 2004 report, GAO recommended that FEMA (1) develop 
and implement standards to help the agency, its contractor, and state and local 
partners collect, analyze, and use mapping data consistently for communities 
with similar levels of risk; (2) develop and implement strategies for partnering 
with communities of varying capabilities and resources; (3) ensure that staff 
levels are sufficient to implement the map modernization program; and, (4) 
develop and implement useful performance measures of stakeholder 
awareness and map use. 5 

Results of Audit 

Approach to Map Modernization 
Could Be Improved 

With congressional funding and support, FEMA has revitalized its program 
for modernizing the nation's flood maps. FEMA has developed a plan 
outlining the priorities and resources needed to accomplish map 
modernization studies in communities across the U.S. However, because of 
budget limitations, FEMA's plan does not reflect user or funding needs. 
Further, while the plan establishes a new mapping quality standard, it provides 
no guidance on how the standard will be achieved. Due to these deficiencies, 



Audit of FEMA 's Cost Estimate for Implementing the Flood Map Modernization Plan, FEMA OIG, Report Number H- 
09-00, September 2000. 

5 Program Strategy Shows Promise, but Challenges Remain, U.S. Government Accountability Office (GAO-04-417, 
March 2004). 
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the plan discourages stakeholder buy-in. More importantly, it may not help 
FEMA meet its map modernization schedule and quality goals. Adjustments 
to the plan and improved FEMA control of costs, schedule, and performance 
would better ensure achievement of program goals. 

Map Modernization Plan Has Limitations 

Office of Management and Budget Circular A-l 1 establishes policies for 
planning and managing investments, such as adequately addressing user, 
funding, and quality requirements, to ensure that investments achieve intended 
benefits. 6 To meet these requirements, FEMA has developed a 
comprehensive plan that outlines priorities, funding, and standards for 
carrying out mapping studies. However, the plan is hindered by budget 
limitations and does not adequately address user or funding requirements. As 
a result, the plan does not ensure buy-in or support for achieving map 
modernization program goals and objectives. 

Mapping Plans and Priorities 

With a budget of $1,475 billion for a six-year period beginning in FY 2003, 
FEMA has been able to move forward with its map modernization program. 
In February 2003, FEMA and mapping stakeholders met at a conference in 
Atlanta, Georgia to determine the factors for prioritizing mapping studies and 
digitization of existing maps. The consensus was to focus on mitigating flood 
risk in areas that had high growth, high population, and a history of significant 
flooding. Participants also suggested that FEMA maximize efficiency by 
conducting basin-wide studies and capitalizing on areas with pre-existing 
mapping data. 

In March 2004, FEMA awarded a single, performance-based contract to a 
primary contractor responsible for executing the map modernization program. 
FEMA charged the contractor with providing architectural and engineering 
services, program management, IT systems development and support, data 
collection activities, customer assistance, and outreach services. While 
FEMA provides centralized program direction and contract oversight, the 
contractor carries out decentralized map modernization activities, such as pre- 
scoping, producing deliverables and work products, and generally supporting 
the map modernization program on a day-to-day basis. To this end, the 
contractor has established field offices to supplement the work of FEMA's ten 
regional offices. (See Figure 2.) 



6 Circular A- 1 1 , Part 7, Planning, Budgeting, Acquisition, and Management of Capital Assets, Executive Office of the 
President, Office of Management and Budget, June 2002. 
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Figure 2: FEMA Regional Offices 

FEMA's regional offices are charged with hiring contractors to conduct 
mapping studies. Alternatively, the regional offices grant funding to mapping 
partners to perform the studies or hire indefinite delivery /indefinite quantity 
contractors to work on their behalf. This arrangement allows FEMA to focus 
on partnering with state and local entities, which ultimately will have 
responsibility for maintaining the updated maps. 

Because the FY 2003 congressional appropriation significantly increased 
funding for the program, map modernization became subject to review and 
approval by senior DHS leadership through its capital planning and 
investment control process. Accordingly, in 2004, FEMA submitted 
documentation to DHS' senior-level Investment Review Board to support the 
program. This documentation included an updated "Exhibit 300" business 
case, required by the Office of Management and Budget to justify a major 
acquisition program. The update was based on FEMA's 1997 map 
modernization plan, the Technical Mapping Advisory Council 
recommendations, and information on innovative technologies and geospatial 
data sharing. In June 2004, DHS' Investment Review Board approved the 
map modernization program, authorizing capability development and 
demonstration to begin. In its approval memorandum, the review board stated 
that an effective implementation plan would be critical to success of the map 
modernization program; and, required that FEMA submit to the Under 
Secretary for Management a schedule for conducting community mapping 
studies. 



7 Developed in accordance with Office of Management and Budget Circular A-ll, Planning, Budgeting, Acquisition, and 
Management of Capital Assets, business cases are designed to ensure that investments are linked to agency missions, 
long-term goals and objectives, and annual performance plans. 
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In response to this direction, FEMA's contractor provided the Multi-Year 
Flood Hazard Identification Plan (MHIP) in November 2004. Built from state 
business plans and assessments of relative flood risk, the MHIP proposes 
budgets and establishes a sequence for conducting mapping studies among 
regions and local communities. The MHIP also communicates the cost, 
schedule, and performance expectations for map modernization activities. 

MHIP Does Not Adequately Address Requirements 

Despite FEMA's efforts to obtain input, the MHIP does not adequately 
address stakeholder needs. FEMA shared copies of the plan with a range of 
audiences nationwide, including the Congress, DHS, and other federal, state, 
and local mapping stakeholders. Also, FEMA posted the MHIP to its web 
site, along with instructions for submitting comments on the plan. According 
to a March 2005 FEMA analysis, the agency received a total of 103 individual 
comments on the MHIP from 31 sources including state, county, and local 
governments, trade associations, and private industry organizations. The 
predominant concern was that the prescribed sequence of mapping studies 
would not provide modernized maps by the required dates. Some 
stakeholders commented that the MHIP strategy would not ensure production 
of accurate and useable maps. Additionally, they asserted that proposed 
budgets and strategies outlined in the MHIP are inadequate to support 
mapping requirements at the state and local level. FEMA noted that it would 
review the comments and make adjustments based on analysis of the feedback 
received. 

Sequence of Mapping Studies 

According to FEMA's March 2005 analysis, 20 of the 31 stakeholders who 
commented on the MHIP were concerned that the sequence of mapping 
studies would not provide modernized maps by the dates needed. Generally, 
the stakeholders requested that mapping studies for their areas be completed 
sooner than scheduled so that they would be better positioned to protect and 
insure against potential flood damage and loss. For example, officials in a 
Minnesota county that lies along a major waterway said that their current 
maps are highly inaccurate and difficult to use to manage flood risk. The 
county requested improved flood plain maps much sooner than the scheduled 
2007 production, 2008 completion, and 2010 adoption dates. Similarly, in 
commenting on the MHIP, the Wisconsin Department of Natural Resources 
submitted proposed changes to the sequence of mapping studies for the state 
along with its business plan. 

Proposed Budgets Are Insufficient 

A number of the stakeholders expressed concern that the proposed budgets 
outlined in the MHIP are inadequate to meet nationwide requirements for map 
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modernization. For instance, an Ohio official stated that the MHIP does not 
reflect the state's true mapping cost requirements. While the MHIP indicates 
that Ohio is to receive between $13 and $15 million for map modernization, 
the state's actual mapping needs are approximately $41 million. This official, 
who was unable to link the MHIP to either the Ohio business plan or the 
state's needs assessment, indicated that FEMA seemed to allocate funding for 
map modernization based on pre-determined amounts rather than detailed 
information provided by the state as input to the MHIP. 

In another instance, Massachusetts flood hazard management officials said 
that although the MHIP identifies $7 million in funding for the state over the 
next five years, about $34 million is needed to modernize flood maps. These 
officials said that the $34 million requirement cited in the state business plan 
is based on careful analysis and reflects the minimum requirement to produce 
accurate maps for areas that already have serious flood hazard problems. 
Further, these officials said that if, in accordance with the MHIP, 
Massachusetts is reduced to mapping the entire state with only $7 million over 
a five-year period, the funding would be so thinly spread that it would only 
cover digitizing existing maps — not conducting new studies to produce 
accurate maps. Massachusetts communities have long expressed 
dissatisfaction with inaccuracies in existing flood maps. They fear going back 
to their communities with digital maps that incorporate the same deficiencies 
that they have known for years. Massachusetts officials concluded that with 
limited funding, they would not be able to generate the information they need 
to regulate new construction and development around flood hazard areas. 

Similarly, New York officials estimated their mapping costs to be $106 
million, based on analysis to support the state's business plan. However, the 
MHIP allotted New York approximately $43 million for mapping purposes. 
State officials believed that the MHIP budget was unrealistic. They said that 
the direction to map the entire state within a five-year period is 
counterproductive and ultimately will result in substandard maps, as they 
would only be able to digitize existing, poor quality maps. In the past, the 
state's philosophy has been to provide highly accurate maps for high-risk 
areas using available funding, and postpone mapping of less critical areas until 
funding becomes available in the future. They said that until release of the 
MHIP, FEMA regional officials seemed to share this point of view. 

In its comments on the MHIP, the Association of State Flood Plain Managers 
concurred that the states' mapping needs assessments are not high-end "wish 
lists" for detailed flood studies and mapping. Rather, association officials said 
that the assessments represent minimum requirements for conducting detailed 
studies in high priority areas alone. They asserted that increased urbanization 
and population density in high-risk areas over the past few decades have 
increased the requirement for more mapping studies and accurate maps. 
Merely digitizing maps leads to a false sense of security among stakeholders 
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as the newer maps, though computerized, will not provide the reliable 
information needed to guard or insure against flood risk. (Figure 3 provides a 
picture of some of the devastation possible due to flooding.) The association 
concluded that while the proposed MHIP budget allocations are not adequate 
to map the nation in five years, they could successfully provide accurate maps 
for the top priority counties in the U.S. — if adjustments are made to the plan. 




Figure 3: Flooding in Asheville, North Carolina after Hurricane Frances. (Photo by Leif 
Skoogfors, September 17, 2004, taken from FEMA's website.) 

Other Complexities of Map Modernization Not Addressed 

Stakeholders believe that the MHIP does not adequately address other 
significant factors needed to achieve FEMA's map modernization goals. 
First, the MHIP does not go far enough to ensure consistent production of 
accurate maps. Previously, there was no clear standard for converting paper 
maps to digital maps. Transferring floodplain illustrations from paper to 
digitized format introduced significant errors. However, with publication of 
the MHIP, FEMA has established "matching best available topography" as the 
minimum standard for producing reliable and accurate maps. Under the new 
standard, maps will no longer include errors that occur when topography is 
not matched, such as streams depicted outside of floodplains or houses at a 
40-foot elevation above the floodplain identified within flood zones. 

Although stakeholders commended FEMA for establishing the new quality 
standard, they expressed concern that the MHIP does not provide guidelines 
for evaluating modernized maps to ensure that the standard is met. For 
example, the MHIP does not provide detailed guidance concerning how to 
select projects for testing, how to conduct testing, the extent to which testing 
is done, and what to do in case of non-compliance. Representatives of the 
Association of State Floodplain Managers said that without such guidance, 
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ongoing mapping studies for FY 2005-2006 certainly would not meet the 
standard. When asked about the lack of guidance for ensuring map quality, 
FEMA officials said that, given other competing program management 
responsibilities, they had not had the time to focus on clearly defining 
methods for ensuring mapping accuracy. However, they recognize the 
importance of publishing the guidelines and currently are soliciting input from 
FEMA regions as to what procedures to include. 

Second, stakeholders said that the MHIP does not present a long-term strategy 
for map modernization. They want to know the scope of the long-term effort 
versus the short-term priorities addressed in the initial effort. They want to 
know FEMA's strategy for mapping the entire nation. They want a clear 
understanding of end-state objectives in map modernization, too. Finally, 
they said that, in the context of a long-range strategy, the MHIP does not 
adequately address costs for map maintenance and re-study. 

For example, officials with the Illinois Association of Floodplain and 
Stormwater Management said that the MHIP should reflect costs for both 
short- and long-term map production and maintenance. A contractor noted 
that the plan alludes to the availability of funding for map and data 
maintenance as a component of engineering and mapping. However, this 
official stated that nowhere in the MHIP does FEMA address the funding 
needed for restudies to maintain the upgraded maps. The contractor said that, 
without such funding, by 2010, the oldest of the modernized map would be 
almost 10 years old. This official cautioned that FEMA must be careful not to 
allow newly modernized maps to become outdated or they will once again 
face the challenges they are currently experiencing. 

In response to these concerns, FEMA officials acknowledged that the map 
modernization program is a long-term effort and that they need a strategy for 
updating existing maps as well as for keeping them current. They noted that, 
within the context of the National Flood Insurance Program, they are 
evaluating resources needed to meet evolving stakeholder requirements over 
the life cycle of flood maps. 

MHIP is Limited by Funding Allocations 

The congressional appropriation in FY 2003 constituted a significant financial 
increase to allow FEMA to move ahead with its map modernization program. 
However, based on comments received about the MHIP, it still was not 
enough to meet stakeholder needs for producing accurate, modernized maps. 
In devising the MHIP, FEMA worked within the congressional funding and 
time limitations to establish sequencing plans and allocate funding to states 
and communities across the nation for map modernization. Although FEMA 
had been in ongoing communication with stakeholders regarding their map 
modernization funding needs and expectations, with respect to the MHIP, 
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FEMA continued to adhere to mapping goals and objectives outlined in its 
1997 mapping plan, with the view that little had changed and the plan 
remained viable. Specifically, FEMA's strategy was to (1) update flood data, 
(2) refit topographic data, and (3) digitize paper maps as much as possible and 
refine those digitized maps at a later date to ensure accuracy. In contrast, as 
discussed above, stakeholders called for more than mere digitization. They 
wanted full funding and detailed plans for creating the most accurate maps 
possible without delay to address their evolving mapping requirements and 
priorities in high-risk floodplain areas. 

The Association of State Flood Plain Managers agreed that FEMA's map 
modernization approach as outlined in the MHIP is not adequate and should 
be modified to address changing funding and flood hazard requirements since 
the original 1997 plan. The association stated that a revised MHIP should 
reflect new information and factors discovered under the initial years map 
modernization. For example, the association said that floods are elevation- 
related phenomena; mapping flood areas requires engineering analysis, 
including hydrology and hydraulic studies, to determine how high waterways 
could potentially rise relative to their surrounding land areas. The association 
said that FEMA must evaluate such new requirements to maximize resources 
for conducting mapping studies. The association also said that modernizing 
the nation's flood maps to meet a new quality standard will take time and 
should not be rushed. Attempting to map the entire nation with limited 
funding in the five-year period outlined in the MHIP, they said, may result in 
chaos. 

MHIP Does Not Ensure Stakeholder Buy-in 

The MHIP in its current form does not encourage stakeholder buy-in and 
support for achieving FEMA's map modernization goals. Because the MHIP 
does not meet their sequencing or funding needs, stakeholders are discouraged 
from taking an active role in the program. If FEMA continues with the MHIP 
map modernization approach as outlined, states and communities may resist 
adopting the resulting maps because they will not view them as accurate or 
reliable to meet critical risk management needs. Merely digitizing maps, 
states fear, may cause them to lose credibility within their communities. 

For example, Massachusetts officials said that the limited MHIP funding and 
digitization approach hinders the state from developing and maintaining 
strong working relationships with its community mapping partners. Similarly, 
New York officials said that the limited mapping approach might damage the 
integrity of the national flood insurance program and perhaps the entire 
community of floodplain managers. Without complete guidance on achieving 



Hydrology is the scientific study of the properties, distribution, and effects of water in the atmosphere, on the earth's 
surface, and in soil and rocks. Hydraulics is the physical science and technology of the static and dynamic behavior of 
fluids. 
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and maintaining minimum quality standards in map production, stakeholders 
lack confidence in FEMA's strategy for ensuring credible maps, too. 

Upon evaluating stakeholder comments on the MHIP, FEMA acknowledged 
that a number of specific flood mapping needs might not be addressed within 
current sequencing plans and funding levels. Nonetheless, FEMA officials 
cautioned that the states' funding requirements and expectations to produce 
highly accurate maps all at once are not realistic, and that map refinement will 
have to be an ongoing process. FEMA reminded stakeholders that the MHIP 
schedules are flexible and that FEMA regions, which control the funding, can 
alter the sequence of mapping studies to meet their individual community 
needs. At the same time, however, FEMA stated that the agency is evaluating 
resources and will continue to work with states and communities to identify 
the most effective approach for meeting evolving stakeholder needs over the 
life cycle of flood maps. 

Program Management Needs Improvement 

Office of Management and Budget Circular A-l 1 directs agencies to employ 
good management disciplines to ensure that programs achieve intended cost, 
schedule, and performance outcomes. FEMA has responsibility for 
monitoring the contractor's map modernization activities. However, the 
agency has not effectively controlled costs and schedules to ensure that the 
contractor is accomplishing the overall program within established budgets 
and timeframes. Further, although FEMA has documented its performance 
management processes, the metrics used are not adequate to provide a 
complete assessment of contractor program management activities. 
Additionally, independent evaluations to ensure quality contractor processes 
and deliverables have not been adequate. 

Program Cost and Schedule Not Effectively Controlled 

FEMA oversight of the contractor's project management and procurement 
activities has not been adequate to control the costs and schedule for the map 
modernization program. In addition, FEMA did not ensure effectively that 
the contractor had the authorizations necessary prior to purchasing IT 
resources on the agency's behalf. 

Project Management System Not Used Effectively 

FEMA has not effectively overseen contractor use of an earned value 
management system for monitoring progress in accomplishing the map 
modernization program. The earned value management system is an 
electronic tool that compares baselined map modernization schedules and 
costs against actual activities and expenditures, applying a formula and 
indicators for determining the ongoing "health" of the program. FEMA and 
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the contractor jointly performed an integrated baseline review to ensure 
complete understanding of the contract requirements and the contractor's 
proposed solutions. As required by the contract, the contractor implemented 
the system at a cost of approximately $2.5 million. 

However, FEMA's use of the earned value management system has not been 
effective because program task managers do not fully know how to use the 
data generated by the system. FEMA managers generally received training in 
earned value management; but, because they were not involved in the initial 
baseline planning, a number of managers did not understand how the budgeted 
costs and schedules for individual line items included in the contract task 
order were developed. Due to their limited understanding, managers did not 
use the earned value management information to the fullest extent possible. 

For example, some managers believed that the earned value management 
reports did not give them adequate information to assess program progress. 
Others stated that they were unclear of the benefits of the information and 
therefore did not use it at all. One FEMA manager did not understand how 
the costs on an invoice related to the work that the contractor had performed 
and therefore requested additional information. In response, the contractor 
provided reports to show how the work performed translated into earned 
value. The manager stated that such reports, coupled with including FEMA 
managers in the baseline planning, will help with earned value management 
processes for future task orders. 

Due to a lack of detailed guidance, FEMA managers used the earned value 
management information in inconsistent ways to carry out their oversight 
responsibilities. For example, one official used an arbitrary 10 percent 
variance from baselined costs and schedules to gauge whether tasks the 
contractor had completed met acceptable levels. Another manager reported 
that, without his knowledge, the contractor had added tasks and costs to his 
contract area of oversight, distorting the original baseline against which to 
assess work performed. Without guidance for using information generated by 
the earned value management system, FEMA managers lack a consistent 
means of evaluating contractor performance and ensuring that the benefits of 
their investments in map modernization are realized. 

Further, the contractor' s monthly reports to FEMA, which accompany the 
earned value management information, lacked specificity. The reports 
summarized program progress, contractor performance, cost and risk analysis, 
and mitigation plans. However, the reports failed to capture key information, 
such as delays experienced by the contractor in developing the Mapping 
Information Platform (MIP) — a web based system for mapping and updating 
flood maps — and the associated impact on the rest of the map modernization 
program. In a March 8, 2005, letter FEMA requested that the contractor 
improve the monthly reports by including the critical information needed to 
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make informed decisions on adjusting the program approach to ensure 
achievement of established goals and objectives. 

Monitoring of Contractor Spending 

Initially, FEMA did not have adequate procedures in place to control the 
contractor's IT expenditures for map modernization. The contract requires 
that FEMA compare each contractor invoice to program requirements, 
budgets, and documented deliverables to determine whether the bill should be 
paid. However, upon reviewing the contractor's December 2004 earned value 
management report, we identified a cost overrun of approximately $1.1 
million for the program management line item in the contract. When 
questioned in this regard, FEMA officials attributed some of the cost overrun 
to contractor purchases of IT equipment that they had not pre- authorized. 

To illustrate, one contractor purchase was for an improved administrative 
system for managing map modernization program documentation. The 
contractor had determined that the previous system was not cost effective to 
maintain, even though a subcontractor had provided it free of charge for the 
first year. The system costs and functionality limitations made continued use 
of the system in subsequent years cumbersome. Therefore, the contractor 
purchased another administrative system at a cost of approximately 
$250,000 — without adequate coordination with FEMA's IT architecture 
office. After learning about the purchase, FEMA managers requested that 
their Applications and Analysis office review the procurement. Review 
officials found that the contractor had purchased the system without an initial 
check to ensure compliance with FEMA's enterprise architecture. FEMA 
reviewed the system to ensure compatibility and suitability, and subsequently 
approved the purchase, even though it stretched an already limited map 
modernization budget. 

Decentralized responsibility for monitoring IT expenditures across the various 
contract line items made such unauthorized purchases possible. For example, 
FEMA officials responsible for overseeing contractor efforts to develop the 
MIP stated that they were not responsible for monitoring IT procurements of 
other contract line items and therefore could not ensure that all purchases met 
FEMA approval. FEMA recently has taken steps to ensure that unauthorized 
contractor purchases do not occur again in the future. Specifically, in 
February 2005, FEMA and the contractor agreed on a new policy for 
reviewing and authorizing IT procurements. The policy states that all requests 
for procurement of IT hardware and software greater than $5,000 must be pre- 
approved by FEMA's Applications and Analysis office. In addition, the 
policy states that all IT acquisitions must meet FEMA's enterprise architecture 
requirements. Since instituting the new policy, FEMA has disapproved 
$305,000 in unauthorized contractor purchases. 
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Performance Management Needs Improvement 



FEMA has not provided the performance management and oversight needed 
to ensure successful accomplishment of the map modernization program. 
Although FEMA has documented its processes for evaluating program 
performance, the performance measures in use are not adequate. Further, 
independent assessments to ensure quality contractor processes and 
deliverables have not been consistent or adequate. 

Performance Measures are Not Adequate 

The Federal Acquisition Regulation requires that agencies institute effective 
approaches for evaluating contractors' execution of performance-based 
contracts. 9 This includes outlining processes for measuring the effectiveness 
of contractor services, deliverables, and work activities. As part of outlining 
these processes, and in coordination with the federal agencies, contractors are 
to develop several specific program management documents, such as 
performance work statements, quality assurance surveillance plans, and 
performance requirements summaries. In addition, the Federal Acquisition 
Regulation calls for the establishment of meaningful measures to assess 
contractor performance in executing specific tasks and activities and to 
support management decisions and actions for sustained progress in achieving 
contract objectives. Such measures help agency managers determine whether 
and in what amount contractors should be awarded for work performed. 

FEMA and the contractor have taken some first steps toward managing 
performance of the map modernization contract. They have worked together 
to document processes for evaluating contractor performance. For example, 
in accordance with the performance work statement, FEMA and the contractor 
have instituted a management system to assess contractor performance in 
seven broad categories that typically drive a program's success or failure: 
leadership; product quality; program schedule; scope and risk management; 
teamwork; and contract management. Further, the contractor has developed a 
matrix that establishes metrics and outlines how contractor performance in 
each of these seven categories is to be measured and scored each quarter. 
Starting in September 2004, about six months after the contract was signed, 
FEMA began using the matrix to assess contractor effectiveness in program 
management and to determine the contractor award fee. 

The measures for evaluating contractor performance as part of this process, 
however, have not been adequate. FEMA and the contractor agreed upon 18 
measures for evaluating the contractor's effectiveness; of these, five measures 
are related to program management. Further, of the five program 



9 Federal Acquisition Regulation, Part 37, General Services Administration, Department of Defense, and National 
Aeronautics and Space Administration, March 2005. 
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management measures, four are event-based: they simply indicate contractor 
compliance in developing a prescribed report or implementing a required 
system. Only one measure was structured for evaluation based on a 
comprehensive assessment of the contractor's performance. The indicators 
for scoring contractor performance in this one measurement area are very 
narrowly focused and do not provide a sound basis for evaluating the 
contractor' s overall program management accomplishments. 

For example, FEMA based the contractor's leadership performance on two 
action items assigned to the contractor at an August 2004 map modernization 
executive steering group meeting. Specifically, the contractor was tasked to 
assess a model for developing the MHIP and review lessons learned from 
DHS' U.S. Visit Program to support development of the MHIP. 10 In 
September 2004, the contractor received a maximum score for leadership, 
based on completion of the two action items but no detailed analysis. 
Similarly, the contractor completed two more action items assigned at a 
subsequent executive steering group meeting the next month. Again, the 
contractor received a maximum score for leadership, although contractor 
actions dealt primarily with examining leadership issues and matters of 
interest rather than actually demonstrating leadership ability. Such high 
contractor scores were based on very narrow categories of performance and 
did not consider the overall "health" of the program under the contractor's 
leadership. Also, the scores were in complete contradiction with a March 
2005 letter in which FEMA discussed cost and schedule overruns and 
criticized the contractor for poor leadership performance in the subsequent 
months of managing the program. 

A FEMA official questioned the agency's approach to collecting, tracking, 
and analyzing the data used to score contractor performance. The official said 
that in a number of instances the measures appeared to be inaccurate. For 
example, this official asked how, in the category of "product quality," the 
contractor could have achieved a high score for having no letters of map 
changes returned for correction when mapping officials had already pointed 
out mapping errors that needed to be addressed. This official noted that no 
one is quite sure how such information is tracked and documented so there is 
no reliable means of verifying the performance measures and scores assigned. 

Further, the official said that some measures appear to have been implemented 
incorrectly. For instance, FEMA was not supposed to assess contract 
management activities until the end of task order one, which was comprised of 
requests for contract modifications and indefinite delivery /indefinite quantity 
contracts. However, FEMA assessed contract management performance 
halfway through execution of task order one and gave the contractor a 



US-Visit is a system that allows the United States to determine the eligibility of foreign travelers to enter the United 
States. 
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"minimum" score, where there should have been no score at all. Similarly, 
the official said that the contractor assigned the maximum score to the scope 
management category in the third quarter of the contract; however, FEMA's 
assessment reduced the score to zero because of no activity. The official 
concluded that potentially scoring contractor performance before services 
have been completed or in areas of no activity results in distorted performance 
assessments. In turn, contractor fees awarded on the basis of such distorted 
performance assessments may be too high, constituting an inappropriate use 
of government funds. To address this issue, the official has requested 
improved guidance for evaluating the contractor's program management 
performance. 

Quality Assurance Activities Have Lagged 

The Federal Acquisition Regulation, part 37, requires that agencies conduct 
quality assurance reviews or inspections to ensure that contractors are meeting 
quality and quantity requirements of performance-based contracts. However, 
FEMA does not have all of the necessary mechanisms in place to adequately 
review work performed under its map modernization contract. For example, 
in accordance with the terms of the contract in 2004, the contractor 
established an assessment office independent of its program office to analyze 
map modernization activities and test new technologies introduced. 

However, due to limited resources, assessment office efforts have focused 
only on testing the web-based MIP system. A subcontractor in the assessment 
office stated that contract negotiations had resulted in a small budget of 
approximately $700,000 and 2.5 testing staff for the office — just enough 
resources to conduct MIP testing. As a result, the assessment office was not 
conducting quality reviews of other contractor deliverables and work products 
that needed them. For example, the assessment office did not evaluate any 
additional technologies, apart from those already used that might assist 
mapping partners in improving processes for producing flood data and maps. 

Until we inquired, a FEMA manager responsible for overseeing the 
assessment office was not aware that these other quality assessments were not 
being performed. Moreover, until we presented copies, this manager was not 
aware that the contractor had prepared quarterly letters to FEMA, which were 
never sent, advising that the agency had not requested any quality inspections 
beyond the MIP testing. For example, in a September 2004 letter, the 
contractor advised FEMA that it had established policies and procedures and 
developed a plan and schedule for conducting inspections in the latter half of 
the contract period, from March 2004 to March 2005. As of December 2004, 
however, FEMA had not requested that such audits be performed. 

Further, FEMA was slow in hiring another contractor, separate from the 
assessment office, to perform independent quality reviews during the critical 
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first months of the map modernization program. The Federal Acquisition 
Regulation, part 37, directs that agencies conduct reviews to provide 
assurance of the quality of contractor work performed. FEMA planned to 
have a systems engineering and technical assistance contractor assess map 
modernization products and services to help reduce the risk of program 
failure. 

However, FEMA did not succeed in bringing the contractor on board until 
September 2004 — six months after the primary contractor began work on map 
modernization. In the interim, the primary contractor had been working at an 
aggressive pace to institute a number of new processes and technologies, 
including the earned value management system, a system development center, 
and the cooperating technical partner data- sharing program — all of which had 
not been assessed. After signing its contract with FEMA in September 2004, 
the systems engineering and technical assistance contractor's quality review 
was limited to products that already had been delivered. Although FEMA was 
not satisfied with some of the primary contractor's work products, the systems 
engineering and technical assistance contractor had no effective means after 
delivery to evaluate the quality of the products or the processes used to 
develop them. The systems engineering and technical assistance contractor 
did not gain the access needed to independently review the delivered products 
until March 2005. 

Coordination Activities Are 
Headed in the Right Direction 

FEMA is moving forward in its efforts to better partner and communicate with 
its mapping stakeholders. However, FEMA could take additional steps to 
maximize the benefits and resources possible through these relationships. For 
instance, although FEMA now works one-on-one with other agencies to 
coordinate mapping activities, FEMA has not instituted the policies, 
agreements, or information sharing mechanisms to effectively support these 
interagency working arrangements. Similarly, while FEMA is building more 
effective partnerships with state and local government entities, all mapping 
stakeholders at this level are not fully involved or aware of key map 
modernization objectives. Further, the call center responsible for addressing 
mapping inquiries does not always provide effective service or accurate 
information, compromising program credibility and potentially placing 
stakeholders and their property at risk. Addressing such collaboration and 
communications issues will be key to ensuring effective use of 
intergovernmental resources, stakeholder commitment, and ultimate success 
in meeting overall map modernization program goals. 
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Progress Made in Coordinating with Other Federal Agencies 

FEMA has made significant progress in its outreach to the federal community, 
working with individual agencies and participating in a number of forums to 
consolidate resources, coordinate activities, and share information to support 
nationwide map modernization efforts. However, FEMA has not put in place 
all of the mechanisms needed to govern and support these interagency 
working relationships. For instance, during the first year of the map 
modernization contract, FEMA did not complete an overarching policy for 
coordinating geospatial data with federal mapping partners. Also, FEMA has 
not instituted the guidelines necessary to govern its working arrangements 
with individual agencies. As a result, FEMA may not be in the best position 
to achieve the cost savings and efficiencies possible through these 
collaborative arrangements. Last, FEMA is not effectively disseminating 
interagency mapping data so that all relevant stakeholders can align with the 
collective efforts of their nationwide mapping partners. 

Guidance Calls for Increased Federal Coordination 

By working together, federal agencies can reduce costs and eliminate 
redundancy in managing geospatial data. According to DHS' Geospatial 
Management Office, geospatial data identifies, depicts, and describes the 
geographic locations, boundaries, or characteristics of inhabitants and natural 
or constructed features on the earth. 

Over the past fifty years, the U.S. Government has issued a range of guidance 
to further federal coordination in collecting and using geospatial data. 
Specifically, Office of Management and Budget Circular A- 16 encourages 
governmental agencies to collaborate on geospatial undertakings. The Office 
of Management and Budget revised the regulation in 1990 to establish the 
Federal Geographic Data Committee, and again in 2002 to reflect geographic 
IT systems enhancements. The Federal Geographic Data Committee is a 19- 
member interagency committee composed of representatives from the 
Executive Office of the President, cabinet agencies, and independent agencies. 
In 1994, Executive Order 12906 made the Federal Geographic Data 
Committee responsible for developing a National Spatial Data Infrastructure, 
which promotes systems interoperability and the pooling of geospatial data 
holdings among federal agencies and their partners. The National Spatial 
Data Infrastructure includes policies, standards, and procedures for 
organizations to cooperatively produce and share geographic data. 
Tremendous growth in geographic information system technologies in recent 
years has helped bring such requirements for coordination of geospatial data 
to the forefront. 
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FEMA is Collaborating with Federal Stakeholders 



FEMA recognizes the need to work with other federal agencies on geospatial 
data activities and has included this requirement in its contract and plans for 
modernizing flood maps. To this end, FEMA participates on the following 
committees and programs focused on geospatial data coordination. 

• Federal Geographic Data Committee : FEMA representatives serve on a 
Coordination Group of this committee, established by Office of 
Management and Budget Circular A- 16. FEMA's guidelines and 
specifications for flood mapping require adherence to Federal Geographic 
Data Committee metadata and accuracy standards, which are incorporated 
into contract costs. 

• Geospatial One-Stop Program : The Department of Interior manages this 
intergovernmental project to support the President's e-Government 
Initiative. Geospatial One-Stop provides an internet portal through which 
government entities and private citizens can access federal and other 
geospatial data collections. The web portal also serves as a focal point for 
identifying similar projects, promoting collaboration, and reducing 
duplicate studies and use of geospatial data resources. Geospatial 
One-Stop representatives work with the Federal Geographic Data 
Committee to ensure that geospatial collection standards are defined and 
maintained. FEMA plans to publish all its flood hazard data updates for 
FY05 and FY06 to the Geospatial One-Stop portal. 

• National Digital Orthophoto Program : The National Digital Orthophoto 
Program, chartered in 1993, is a "consortium of Federal agencies with the 
purpose of developing and maintaining national orthoimagery coverage in 
the public domain by establishing partnerships with Federal, State, local, 
tribal, and private organizations." 11 As part of this program, FEMA is 
developing a registry for posting planned investments in elevation and 
orthoimagery used on maps. Data from the registry will be pulled by the 
Geospatial One-Stop portal and included on that site. 

• National Digital Elevation Program : FEMA is using the same registry 
cited above to support the National Digital Elevation Program. Federal 
agencies created this program as a central point for establishing 
coordination policies, procedures, and standards for sharing elevation data 
across federal, state, local, and private entities. 

In addition, FEMA has had the opportunity to work with agencies on a one- 
on-one basis. For example, FEMA coordinates with the National Oceanic and 



1 1 Digital orthophotography refers to uniform-scale photographs of landmasses, which can be used to measure distance 
and can sometimes serve as base maps. 
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Atmospheric Administration's National Geodetic Survey to move its mapping 
work from the former National Geodetic Vertical Data standard of 1929 to the 
new, federally-mandated North American Vertical Datum standard of 1988. 
The new standard allows for greater accuracy and systems interoperability in 
flood mapping products. FEMA is also working closely with the U.S. Army 
Corps of Engineers and the U.S. Census Bureau to identify opportunities to 
coordinate on existing geospatial data holdings as well as upcoming projects. 

Additional Steps Needed to Support Collaboration Activities 

While FEMA is making progress in coordinating with and using the 
capabilities and data sets of other federal agencies, several challenges remain 
to full and seamless information sharing. For example, FEMA has not yet 
finalized its overarching policy for coordinating geospatial data with its 
mapping partners. FEMA representatives stated that they started to draft the 
policy prior to awarding the contract to the primary contractor. The contractor 
was to complete work on the policy and resubmit it for agency approval. 
However, after the contractor submitted the draft to FEMA, the agency 
responded with a discrepancy report indicating that the contractor' s product 
was "significantly worse than the original draft." FEMA directed the 
contractor to revisit the document to better develop procedures for working 
with federal and state entities and ensuring compliance with all relevant 
regulations and policies. As of July 2005, FEMA's data coordination policy 
had yet to be finalized. 

In the absence of a final data coordination policy, mapping partners lack 
formal guidance for mapping data collection and use, and may inadvertently 
create products of little value to other federal agencies. The rework to correct 
deficiencies with the mapping products may result in redundant cost and 
effort. Additionally, as one FEMA representative stated, the agency's delay in 
releasing a finalized data policy compromises its credibility with mapping 
stakeholders. 

Additionally, FEMA has not put in place all of the guidelines needed to 
govern specific interagency working arrangements. FEMA uses memoranda 
of understanding to define roles and responsibilities in collaborating with 
other federal agencies on flood mapping. FEMA has drafted such agreements 
for working with both the U.S. Geological Survey and the National Geodetic 
Survey; however, the documents have not yet been finalized. 

When asked why the memoranda have not been completed, FEMA officials 
said that confusing guidance has hindered efforts to put the documents in 
place. As a DHS component, FEMA must adhere to the policies and 
standards of the department's Geospatial Management Office. The office, 
established through provisions of the Intelligence Reform and Terrorist 
Prevention Act of 2004, is responsible for establishing and maintaining 
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appropriate relationships with federal, state, tribal, local, and private sector 

12 

organizations on geospatial management technology matters. DHS 
Management Directive 4030 defines the roles and responsibilities of the 
office, stating that the office is to provide departmentwide leadership and 
guidance to ensure proper coordination with other federal agencies and 

13 

compliance with all applicable laws. By September 2007, the office is also 
to provide oversight of all geospatial IT systems management, procurement, 
security, and interoperability issues at DHS. 

However, since the Geospatial Management Office was established, its 
management has used a "do no harm" approach — leaving legacy agencies 
within DHS, including FEMA and its map modernization program, to manage 
as they deem appropriate. As a result, map modernization managers, unsure 
whether they should act on their own authority or work through the Geospatial 
Management Office, are waiting to take action to finalize the agreements. 
Until the finalized agreements are in place, whether departmentwide or 
agency-derived, there is no assurance that FEMA and other agencies will be in 
accordance with one another. FEMA also might miss significant 
opportunities for increased collaboration in map modernization efforts. 

Further, although FEMA receives periodic updates regarding a central 
repository for federal mapping information, the agency has not devised a 
means of disseminating this information to the mapping partners that need it. 
Specifically, the U.S. Census Bureau maintains the Topologically Integrated 
Geographic Encoding and Referencing Enhancement Database, which 
provides an inventory of mapping data, including contact information, web 
addresses, agency names, and other related information. FEMA's regional 
offices and contractors rely upon such information to support their mapping 
studies nationwide. However, FEMA is still working on a plan on how to get 
the information out on a timely basis for nationwide reference. Where 
regional offices and contractors do not obtain the updated information on 
available data sets, they may perform unnecessary mapping studies, resulting 
in redundant effort and resource expenditures. Further, communities 
undertaking activities to identify mapping needs and the resources available to 
fulfill them, may unknowingly exclude information that could have helped 
produce more accurate maps. 

Progress Made in Coordinating with State and Local Entities 

As with the federal level, FEMA has taken steps to strengthen 
communications and outreach to its state and local mapping partners. 



Intelligence Reform and Terrorism Prevention Act of 2004, Public Law 108-458, December 17, 2004. 
13 Geospatial Management Office, MD 4030, Department of Homeland Security Management Directive System, 
November 12, 2004. 
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However, additional steps are still needed to effectively articulate map 
modernization objectives and consistently support stakeholders at this level. 

Steps Taken to Strengthen State and Local Partnerships 

In accordance with recommendations of both the Technical Mapping 
Advisory Council and the GAO, 14 FEMA is working to build constituent 
interest and support for modernizing flood maps by using a process that 
includes public education, outreach, communication, and establishment of 
partnerships with its state and local government stakeholders. FEMA's 
strategy is to maximize the mapping resources and experience of these 
stakeholders. The MHIP outlines FEMA's vision for such intergovernmental 
cooperation and coordination to achieve map modernization goals. 

As discussed in the MHIP, FEMA has established a Cooperating Technical 
Partner program to co-opt state and local participation in its map 
modernization program. Cooperating technical partners sign agreements with 
FEMA that clearly define objectives, standards, milestones, and quality 
controls for activities needed to develop or modernize mapping products. At 
the close of FY 2003, FEMA had 151 mapping partners. As of March 2005, 
the agency had approximately 208 partners, 193 of which have mapping 
agreements in place. In FY 2004, these partners collectively provided about 
$62 million leveraged dollars to support FEMA's flood map modernization 
program. FEMA defines leveraged dollars as the products — field surveys and 
topographic data — and assistance that add quality to mapping products. 

As the map modernization program continues to grow and expand, FEMA is 
working to extend its technical partnership base and the program' s overall 
reach. Specifically, FEMA is moving toward multi-jurisdictional cooperating 
technical partner agreements, which consolidate the resources of smaller 
mapping partners into larger cooperative agreements to ensure more unified, 
seamless partnerships. The State of Georgia, Department of Natural 
Resources, for example, has entered into a cooperative agreement with FEMA 
to pool resources and coordinate mapping activities on a statewide basis 
versus working with individual counties or municipalities. State officials cite 
delegation of coordination responsibility to each state as a whole as one of the 
greatest advantages of this type of agreement, because the states have better 
relationships with individual counties than FEMA does. Multi-jurisdictional 
cooperative technical partnerships also encourage integration of various 
organizations, resulting in additional resources for flood mapping. 

Further, cooperative agreements with several state and local level forums 
bolster FEMA's communication and outreach capabilities. For example, to 
better communicate its map modernization vision and goals as well as to 



GAO-04-417, March 2004. 
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coordinate activities with stakeholders nationwide, FEMA sponsors annual 
conferences of the Association of State Floodplain Managers and regularly 
sends representatives to symposiums led by the National Association of Flood 
and Stormwater Management Agencies. Also, FEMA has entered into a 
cooperative agreement with the National States Geographic Information 
Council. Under this agreement, FEMA coordinates with the Council to 
identify and leverage high-resolution light detection and ranging technology 
and orthophotography available throughout the nation. 15 These are among the 
most costly of all mapping technologies, as the cost to obtain light detection 
and ranging technology to map the entire nation exceeds $1.7 billion, while 
the cost for orthophotography approximates $880 million. By working with 
the Council, FEMA can significantly streamline efforts and reduce costs to 
produce mapping products. 

Written materials supplement FEMA's communications with state and local 
entities. For instance, as discussed above, the MHIP supports outreach efforts 
and also serves as a guide for stakeholder planning, scheduling, and use of 
resources in map modernization. Further, FEMA provides standard operating 
procedures for the regional offices and the contractors that support them. 
Though initially some contracting staff reported difficulties working with their 
regional office counterparts, FEMA has taken steps to improve these working 
relationships. For example, FEMA has established agreements in each of its 
ten regions, outlining specific responsibilities of the regional offices versus 
those of the contractors. Additionally, the contractors now participate in bi- 
weekly conference calls with their respective regional offices to discuss 
concerns and provide progress reports on mapping activities. 

Room For Communication Improvement Exists 

While FEMA continues to make great strides in its partnership-building and 
outreach efforts, several challenges remain. Stakeholders do not fully 
understand the goals, objectives, and standards that must be adhered to in map 
modernization. Further, the contractor's call center is not functioning as 
effectively as it could to support mapping stakeholders. 

Map Modernization End Goals Not Effectively Communicated 

As discussed above, the constrained funding and schedules for conducting 
mapping studies, as outlined in the MHIP, have led to the perception among 
stakeholders that instead of updating and computerizing mapping products to 
ensure accuracy, FEMA is simply digitizing outdated paper maps. When we 
asked the map modernization program manager about the digitization issue, 
this official said that such beliefs are not entirely accurate. While the MHIP 



" Light detection and ranging technology identifies distant objects and analyzes pulsed laser light reflected from their 
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may not effectively communicate the full strategy, FEMA officials concede 
that FEMA is not redoing floodplain studies for the entire nation. Re-studies, 
they said, are costly and the program does not have enough money to re-map 
the entire nation. Officials emphasized, however, that bad information would 
not simply be digitized. Rather, in most cases the agency is using 
"redelineation" — a process involving the redrawing of floodplain boundaries 
using more detailed or current topographical information and existing flood 
elevations. While redelineation does not generally result in significantly 
updated flood elevations, FEMA officials believe that it is a good first step. 

The FEMA program manager acknowledged that misperceptions are the result 
of a lack of effective communication and outreach from headquarters and 
regional offices to stakeholders. This FEMA manager acknowledged that the 
program must work to improve communication strategies, adding that FEMA 
and the contractor could have done a better job early on to coordinate outreach 
activities. 

Similarly, FEMA's communication and outreach to stakeholders regarding 
elevation data standards need improvement. In 1993, the Congress mandated 
the North American Vertical Datum of 1988 (NAVD 88) as the official 
standard for the federal government to use to measure surface contours and 
base flood elevations, identifying which properties and locations are in flood 
zones and which are not. 16 However, FEMA is not requiring that its mapping 
partners use this standard when converting their mapping products from paper 
to digital format. In fact, FEMA program officials recently provided an 
update on map modernization status showing that only 22 percent of all 214 
mapping studies identified had been performed in NAVD 88, while the 
remaining 78 percent remained in the former National Geodetic Vertical 
Datum of 1929 (NGVD 29) standard. 

As mentioned earlier, FEMA has successfully coordinated use of the newer 
elevation standard at the federal level. One particular community, Fort 
Collins, Colorado, decided to change from the older datum after FEMA and 
the National Geodetic Survey reached out to inform it of the many issues 
facing the old datum. In general, however, FEMA's regional offices have 
placed limited focus on communicating to individual communities the 
importance of converting to the new standard. Consequently, at the regional 
level, a number of communities are opting to remain in NGVD 29 because it 
is less costly to do so. 

While the costs of converting to the newer elevation standard may seem 
prohibitive to some communities, there are also significant benefits that 
should not be overlooked. These benefits include greater accuracy in mapping 



"The Federal Register (Vol. 58, No. 120, page 34245, June 24, 1993) affirmed NAVD 88 as the official vertical datum 
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studies and comparability of layers (specifically, flood layers as they relate to 
other multi-hazard layers). Conversely, there are significant drawbacks to 
communities remaining in the older elevation standard. For example, 
according to an official with the National Geodetic Survey, NGVD 29 is not 
compatible with new global positioning systems currently used to specify map 
layers. Systems using the old NGVD 29 standard likely will not be 
technically supported in the coming years. More importantly, mapping 
products that blend the two data formats result in miscalculated base flood 
elevations. Without accurate base flood elevation data, properties may be 
incorrectly located outside of flood zones, placing both the properties and 
their residents at risk. Stakeholders who have been well informed of the 
benefits of the new standard and the drawbacks of the old are more likely to 
opt for conversion, as in the case of Fort Collins, Colorado. 

Non-Cooperating Technical Partners Overlooked 

In addition to the need for defined modernization goals and adherence to the 
national elevation standard, we are concerned about the level of 
communication offered to non-cooperating technical partners. Less advanced 
communities may simply be unaware of FEMA's map modernization efforts. 
For example, two industry officials stated that FEMA regions, while playing a 
greater role in map modernization than in the past, generally hire contractors 
to perform modeling and other mapping activities. They stated that these 
contractors, who serve primarily as engineers, generally are not known for 
outreach, often leaving communities uninformed of map modernization 
efforts. 

One county that we visited reported having "very little involvement with the 
map modernization program," even though the county was listed in the MHIP 
as a mapping project for FY 2003 and FEMA was already in the process of 
completing work on the county's flood maps. During our visit, however, we 
found that the county, which had been using the same paper maps since 1981, 
was unaware of FEMA's map modernization efforts on its behalf. When 
questioned about the lack of information provided to the county, FEMA 
representatives stated that the previous mapping contract used to produce the 
county wide platform, coupled with intense pressure to scope and award new 
FY 2004 map modernization project dollars, resulted in ineffective regional 
office tracking and communication with the county. 

After being informed of the oversight, FEMA quickly reengaged with this 
particular county, meeting with county officials and other affected 
communities in the area in December 2004. Additionally, FEMA delayed 
issuance of the preliminary flood map for the county in order to accommodate 
and potentially incorporate any additional data that the county or its 
incorporated communities might provide. The end result is an informed 
community with significant time to include the necessary letters of map 
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change and other mapping data for FEMA's consideration. Although FEMA 
deems this an isolated incident, if we had not identified the oversight, new 
mapping data would not have been included in the county's final mapping 
product, possibly compromising data integrity and resulting in a map that 
inaccurately depicts flood zones. 

Call Center Operations are Ineffective 

Effectiveness of service provided by FEMA's Map Assistance Center — 
known as the call center — is another area where improvement is warranted. 
The call center, designed to provide flood mapping information and assistance 
to the public, is charged with answering telephone inquiries on a variety of 
issues such as letters of map change, document order forms, FEMA 
publications, and frequently asked questions. The call center logs 
approximately 9,000 calls per month. According to FEMA brochures and 
communication materials, the call center will provide support to MIP web site 
users in the future, too. 

The call center is an integral component of FEMA's map modernization 
program. However, the agency has encountered problems with the center 
since transitioning from the contractor formerly responsible for its operations. 
The new contractor succeeded in establishing the call center and making it 
fully operational two days in advance of the deadline. But, after closely 
monitoring call center performance over a number of months, FEMA 
determined that the center was not meeting its goals of providing effective 
service to callers. 

Specifically, as part of its monitoring process, FEMA representatives 
randomly selected calls for review each day to determine whether responders 
were accurately and effectively providing information. Two days in a row, a 
FEMA representative found mistakes in all calls reviewed. In one instance, a 
call center operator took nearly eight minutes to respond to a caller, ultimately 
providing an incorrect map date. In another instance, a responder made no 
attempt to identify or provide useful details to an inquirer about a letter of map 
change. In a third case, a respondent thoroughly confused and frustrated a 
caller. When the caller voiced his frustration at the conclusion of the call, the 
respondent did not offer further assistance. 

In discussions with us, map modernization program officials speculated that 
problems experienced with the call center were likely the result of 
inexperience at three different levels. They stated that the contractor, 
unfamiliar with the needs of a mapping assistance call center, may have 
overlooked the general needs of the facility. They said that inexperience on 
the part of the call center operators — college students who knew little about 
flood mapping — led them to provide inaccurate information to callers. 
Further, the performance-based contracting vehicle allowed FEMA only to 
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dictate the desired product (that is, the call center), but not the necessarily the 
processes or procedures for achieving the end result. 

FEMA allowed the contractor until January 2005 to improve call center 
operations. The contractor, in turn, hired a third party organization that 
evaluated call center service and provided a report to FEMA in February 
2005. First, the report confirmed that the center's operators were providing 
inaccurate information (estimated as high as 30 percent) to callers — a practice 
which could potentially place individuals at risk if they were not well- 
informed about mapping policies and regulations, flood zones, or the need for 
flood insurance. Second, the report found that the center's voice recognition 
system was less than user-friendly, frustrating and alienating callers and 
compromising commitment to mapping program objectives. 

As a result of the report, the contractor outlined steps to reorganize and 
improve center operations, including simplifying the voice recognition system 
and allowing callers to press the "zero" button at any time to access a live 
operator. Additionally, the contractor plans to move "tier 1" call center 
operators to the contractor's headquarters facility in Alexandria, Virginia, 
where operators can obtain better training and assistance by subject matter 
experts. The contractor will implement a new e-mail tracking system to better 
support center operators. Finally, the contractor will continue monitoring 
calls to ensure sustained center performance in providing timely and accurate 
service. 

Mapping Platform and Tools Are At Risk 

As part of its map modernization efforts, FEMA is developing a web-based 
technology platform that will support the management, extraction, sharing, 
and efficient production of digital maps. However, progress has been limited 
because the system development approach is not well executed. Specifically, 
a lack of clear contractor expectations, underestimation of the scope and 
complexity of the project, an overly aggressive development schedule, poorly 
defined system requirements, and an ineffective system development 
approach have led to significant development and implementation delays as 
well as cost overruns. As a result, the mapping platform may not achieve the 
functionality and cost savings anticipated. 

Expectations for Mapping Information Platform 

The MIP — a premier, web-based, data collection and delivery system — is a 
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key component of FEMA's map modernization program. FEMA is 
investing over $30 million to develop and implement the system, which is 



FEMA defines a premier system as a state-of-the art, interoperable, continuously evolving, living, dynamic, and 
integrated system. 
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intended to facilitate the management, sharing, and production of digital 
mapping data. FEMA expects that the new system will achieve its goal of 
reducing the time it takes to produce new digital flood insurance rate maps. 
Also, the MIP is to provide reliable flood and multi-hazard data to support 
stakeholders — from federal and state mapping partners to individual 
homeowners — in their risk management decisionmaking. Because the system 
will be available through the internet, users will be able to quickly access the 
information needed and take appropriate actions to reduce vulnerability to 
damage or loss resulting from potential natural disasters. 




Figure 4: Mapping Information Platform 



The MIP will feature a number of technology components that will assist with 
map modernization project management. Specifically, the MIP will apply 
leading industry practices by including a project tracking application to help 
managers document and monitor activities, schedules, budgets, and 
expenditures for mapping studies and revisions. The MIP also will include a 
digital "dashboard" — another project management tool — for monitoring 
mapping study and digitization processes from inception to completion. The 
digital dashboard uses a green-yellow-red coding scheme to indicate the status 
of an initiative and automatically notifies the appropriate managers when 
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studies are off track. Further, the MIP will host several software tools to 
assist FEMA partners and mapping contractors in producing and editing 
digital flood map panels. The MIP will include online user training for each 
component of the overall mapping platform. FEMA has set out specific time 
frames for delivery of the MIP. The performance work statement for one of 
the map modernization contracts, signed in March 2004, requires that the 
contractor deliver several major components of the MIP in three releases over 
a one-year period, as follows: 



In this color coding scheme, green indicates that the study is proceeding according to plans, yellow indicates schedule 
slippage or other issues regarding product delivery, and red means that the study is in jeopardy and unlikely to achieve 
objectives. 
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• Release 1 : By June 2004, consolidate existing digital flood map 
repositories and establish the MIP on the internet along with capability for 
users to access and view flood maps and other geospatial information. 

• Release 2 : By September 2004, provide FEMA with the digital dashboard 
and other tools for tracking progress regarding mapping studies and letters 
of map changes. Additionally, the contractor was to provide through the 
MIP several highly specialized tools and technologies to support map 
development: the Watershed Information System and Digital Flood 
Insurance Rate Map Production Tools. 

• Release 3 : By March 2005, provide additional capabilities for more 
complex mapping activities, such as data modeling, storing the 
intermediate data used to create flood maps, providing data security, and 
ensuring mapping data quality. 

Platform Deployment Has Experienced Schedule and Cost Overruns 

While some progress has been made in developing the MIP and the map 
production tools, the contractor has not met several deadlines and the platform 
is significantly over budget. Specifically, although the contractor met the 90- 
day time frame for delivering release 1 of the MIP, current estimates indicate 
that completion of release 2 will not be deployed fully until late July 2005 — 
about 10 months behind the originally scheduled date of September 2004. 
The delays began after the contractor identified the unexpected need to 
provide additional project tracking capabilities to release 2 deliverables to 
better meet FEMA's requirements. In order to provide the additional 
capabilities without delaying delivery of the mapping tools, the contractor in 
October 2004 suggested dividing release 2 into sections: release 2.1 would 
launch the two mapping tool applications, while release 2.2 would deploy the 
management tracking tools. FEMA concurred with this proposal. 

However, this newly revised release 2 strategy did not work as planned. In 
November 2004, when the contractor provided the mapping tools — the 
Watershed Information System and the Digital Flood Insurance Rate Map 
software — comprising release 2.1, FEMA rejected the deliverables. FEMA 
contract managers stated that the tools did not satisfy the requirements 
outlined in the performance work statement. For example, they said that the 
release did not include the full suite of Digital Flood Insurance Rate Map 
tools. The deployment plan also did not identify the main audiences for the 
tools, how the tools should be used, or how users would be trained and 
supported. In addition, the contractor had not provided performance test 
results for the Watershed Information System tool. 

To further adjust the release 2 delivery schedule and provide users with at 
least one of the tools with which to begin mapping studies, FEMA officials 
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decided to deploy the Watershed Information System, contingent upon 
acceptable system performance testing. Accordingly, the contractor further 
delineated the delivery schedule, identifying the Watershed Information 
System as release 2.1 and the Digital Flood Insurance Rate Map production 
tools as release 2.1.1. After much discussion and several contract 
modifications, the contractor successfully launched the Watershed 
Management Tool in January 2005 and FEMA approved the Digital Flood 
Insurance Rate Map production tools in May 2005. FEMA expected the 
contractor to deliver MIP release 2.2 and the project management tracking 
tools, beginning in June- July 2005. 

Because they build upon MIP release 2.2, the subsequent deliverables for 
release 3 will be delayed, too. Originally scheduled for March 2005, release 3 
now is not expected until late September 2005. Although the contractor 
provided the requirements document for release 3 to FEMA in February 2005, 
FEMA rejected the document, stating that it did not include sufficient detail. 
For example, a FEMA official who participated in Joint Application 
Development sessions recalled discussing with the contractor functions to 
include in release 3, such as storing and validating the data compiled from 
mapping studies. The FEMA official reviewed the contractor's requirements 
documentation, but noted that such details had not been included. FEMA 
accepted the requirements in May 2005 after the contractor made the 
necessary changes. Nonetheless, due to the repeated revisions to the 
deliverables, release 3 will be about six months behind schedule. 

These schedule delays have had an adverse impact on the cost to produce the 
MIP. The original budgeted cost to complete the MIP was $27.7 million. 
However, as of April 2005, FEMA had already spent $32.1 million to acquire 
release 1 and parts of the subsequent releases. As of May 2005, FEMA was 
still waiting for the contractor to provide an estimate of the cost to complete 
releases 2.2 and 3. One official estimated that the total cost to complete the 
MIP could range from $17 to $20 million. If such an estimate holds true, the 
total project cost could exceed $52 million — approximately $24 million over 
budget. 

IT Approach Not Well Coordinated 

MIP schedule and cost overruns may be attributed to a number of issues that 
FEMA has not managed effectively. These issues include unclear contractor 
expectations, an overly aggressive system development schedule, 
underestimation of the project scope and complexity, inadequate system 
requirements definition, and an ineffective systems development approach. 
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Contractor Expectations Not Well Defined 



Office of Management and Budget guidance for performance-based 
contracting states that performance work statements should be written using 
clear and precise wording and task-oriented statements. 19 However, FEMA's 
agreement with the contractor on MIP development plans and expectations 
was unclear, contributing to the schedule and cost overruns discussed above. 

For example, the performance work statement for developing the MIP 
contains such vague language that FEMA and the contractor each interpret it 
differently. In response to FEMA's statement of objectives concerning map 
modernization, the contractor developed the performance work statement, 
describing services to be performed under task order one of the contract. 
However, the work statement includes vague terms, such as "thin-thread" and 
"thick-thread," referring to the level of capability that the contractor planned 
to provide in the various MIP releases. Loosely defined, "thin-thread" 
implied less capability while "thick-thread" meant more robust capability. 
The work statement does not define the terms, allowing the contractor 
considerable latitude as to what to deliver, and creating the possibility that the 
contractor's work would not satisfy FEMA's expectations. Such a 
discrepancy occurred when the contractor provided FEMA with a description 
of the functionality that would be included in MIP releases 2 and 3. 

Specifically, in September 2004, FEMA learned that the contractor had not 
included in MIP development the ability to track completion of mapping 
studies on a basin-wide basis. By congressional mandate, FEMA is required 
to study an entire geographical area drained by a river and its tributaries; 
however, the contractor had planned to track study progress on a larger 
countywide basis instead. After identifying this deficiency, FEMA reminded 
the contractor that, according to the performance work statement, MIP 
functionality was supposed to be similar to that of the Monitoring Information 
on Contracted Studies System, its predecessor, which could track mapping 
studies by basin. In reply, the contractor asserted that basin- wide studies were 
not within the scope of MIP release 2 and 3 and that such "thick-thread" 
capability had not been envisioned for this phase of MIP development. 
FEMA continued to disagree. Ultimately, in October 2004, the contractor 
agreed to include the capability for tracking basin- wide studies in MIP 
development, but at additional time and cost. 

In November 2004, FEMA identified another misinterpretation of the 
performance work statement requirements when the contractor attempted to 
deliver web-based tools for producing digital maps. In this instance, FEMA 
rejected the delivery, stating that the contractor had not provided the full suite 



A Guide to Best Practices for Performance-Based Service Contracting (Final Edition), Office of Federal Procurement 
Policy, Office of Management and Budget, October 1998. 
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of tools needed for the 30 or more steps required to complete a digital flood 
insurance rate map. In response, the contractor claimed that the initial 
delivery was never intended to include all of the tools for completing all steps 
in the map production process. Further, the contractor stated that its contract 
with FEMA did not include adjustments to reflect new delineations in MIP 
release 2 development. Ultimately, the contractor agreed to provide the full 
suite of tools, but again at additional cost and with added time for delivery. 

MIP Development Scope and Complexity Underestimated 

Despite federal requirements for clear and realistic expectations in 
performance-based contracting, the contractor underestimated the scope and 
complexity of MIP development. Specifically, early in the MIP development 
process, the contractor encountered a number of unanticipated technical 
difficulties that resulted in delays in starting work on MIP release 2. The 
contractor summarized these difficulties in an August 2004 letter to FEMA, 
outlining problems in improving map server scalability to provide additional 
layers of data on the MIP portal. The contractor had to reallocate MIP 
development team resources to address this issue. 

In addition, while conducting infrastructure prototyping, the contractor 
identified technical integration issues among several of the middleware 
software products. The contractor noted that the system architecture had to be 
changed to accommodate differences between business workflow and portal 
software requirements. The architectural change required additional time to 
integrate the middleware software products. 

Further, the contractor had to expend considerable time and resources to 
understand the mapping study process as a prelude to developing functional 
requirements for the MIP software. In examining this business process, the 
contractor found interdependencies between two contract line items: IT and 
data management and flood engineering and mapping. Time to fully 
understand and map out these interdependencies slowed development of the 
functional requirements and extended the MIP development schedule. 

According to FEMA officials, they since have instituted processes to provide 
improved oversight and responsiveness to resolve scope and technical issues. 
For example, the contractor provides FEMA with a detailed weekly briefing 
on the status of the MIP. This briefing describes areas of risk and is a forum 
to discuss scope questions and uncertainties. The meetings also use detailed 
schedule and resource plans to ensure that the program is on schedule and 
within budget. If scope issues are raised that cannot be resolved within these 
constraints, FEMA will use change control procedures it has instituted and 
modify the program scope, budget, and schedule accordingly. Also, FEMA's 
independent quality control reviewer now is involved in daily status meetings 
with the contractor to review processes and raise issues for FEMA's 
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resolution. Additionally, an Engineering Review Board now convenes every 
two weeks to review software patches and changes in the production 
environments. This formal review process helps ensure that changes to the 
system are reviewed, recorded, and approved by cross-functional FEMA 
representatives, such as engineering, mapping, and IT officials. 

Aggressive MIP Development Schedule 

The contractor's proposal to develop the MIP software within one year was 
ambitious as well, leaving little room to accommodate unexpected delays. 
Specifically, in conjunction with the technical problems discussed above, the 
contractor experienced difficulties in staffing the system engineering and user 
interface teams. These efforts took longer than expected, delaying the start of 
system architectural and design activities and pushing the product delivery 
schedule out beyond the original September 2004 deadline. 

In efforts to meet the deadline, the contractor began writing MIP software 
code and analyzing system requirements concurrently, instead of sequentially 
in accordance with standard practice. Nonetheless, the contractor's delivery 
of the completed software code for MIP release 2 in December 2004 was still 
late. After FEMA realized that the system did not include the necessary 
functionality — possibly due to the incomplete requirements analysis — the 
contractor had to expend additional time and money to correct the software 
code. This pattern of system development continued for release 3 as well. 
Although FEMA did not formally approve the requirements document for 
release 3 until May 2005, by that time the contractor had already completed 
90 percent of the program code development. 

A FEMA representative questioned how the contractor could be developing 
MIP release 3 when release 2.2 upon which it was to build had not yet been 
completed. Such code development prior to requirements agreement, this 
official reasoned, poses the risk of the system not meeting user or functional 
needs. When this happens, significant recoding may be required, leading to 
additional costs and delays. 

MIP Requirements Not Well Defined 

Office of Management and Budget Circular A-l 1 directs agencies to reduce 
project risk by involving users in the design of IT assets. However, at the start 
of the contract in March 2004, FEMA and the contractor did not coordinate 
effectively to identify user and technical requirements for the MIP. Several 
FEMA officials stated that there was little communication between FEMA 
and the contractor on these issues during an initial system development 
discussion. The contractor had already begun work to develop the system 
prior to the meeting and therefore had not adequately considered 
requirements. By the time that FEMA and the contractor signed off on the 
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requirements in January 2005, the contractor had already completed the 
system development. Release 3 requirements initially lacked detail, too; 
FEMA and the contractor have worked closely to try to resolve these issues. 

In addition to this initial lack of communication, FEMA's quality control 
contractor found that the process that the contractor used to document 
requirements was inadequate. Best practices in requirements management 
include providing a requirements baseline, traceability matrix, and effective 
change controls. In line with these practices, a requirements management 
plan developed by the MIP contractor discusses the links between user needs 
and system capabilities, along with processes for configuration management 
throughout the software development effort. 2 The plan describes the various 
types of user and technical requirements, their respective features, what 
should be documented, and how this information should be used to manage 
requirements, too. Although the contractor considered this plan an internal 
work product and not part of any deliverable to FEMA, the contractor posted 
the plan to an electronic bulletin board, making it available for review by 
FEMA's quality control contractor. 

The quality control contractor reviewed the MIP requirements management 
plan and found that, while the plan lists activities essential to requirements 
management, it lacks specificity. For example, the plan does not reference a 
documented requirements baseline and only superficially addresses issue 
tracking — the process for resolving discrepancies. Further, change control 
processes, policies, and procedures are not clearly outlined. There is no 
mention in the plan of the need for FEMA to approve the requirements 
identified by the contractor responsible for developing the MIP. The quality 
control contractor consequently made ten recommendations for improving the 
requirements process. FEMA officials said that they have shared these 
findings and recommendations with the MIP developer. 

System Development Methodology Not Applied 

Leading organizations use standard methodologies for systems development. 
Likewise, the contractor' s performance work statement notes that 
development of each MIP release should follow Software Engineering 
Institute Capability Maturity Model guidelines, which require thorough 
system documentation, review at each phase of the system life cycle, and 
testing by the developer and independent verifiers prior to product delivery. 
Ultimately, the final product is to be subject to acceptance testing to ensure 
that all user and functional requirements are met. 

However, FEMA officials were concerned that the contractor did not follow 
such guidelines in providing mapping tools for the MIP. Specifically, based 
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on a review of the contractor's assessment of MIP release 2.1, FEMA's 
quality control contractor affirmed that capability maturity model processes 
had not been employed at each stage of the life cycle in developing the 
mapping tools. 21 The quality control contractor said that testing against 
established requirements for release 2.1 had not been not adequately 
performed or documented; and, software code had been modified and moved 
between the simulated test and production environments in the middle of 
testing. Several significant defects found in the product upon final user 
acceptance testing confirmed that the developer's initial tests had been 
inadequate. 

According to FEMA officials, the contractor has improved weekly status 
reporting to the agency. The program schedules provided in the status reports 
incorporate the system life-cycle development process and identify specific 
tasks for each phase of the life-cycle, helping FEMA ensure that processes are 
followed and tasks are not skipped. Additionally, FEMA has asked its quality 
control contractor to take an active role in evaluating areas that need process 
improvements. For example, the quality control contractor recently reviewed 
and reported to FEMA on deficiencies in the contractor's processes for 
performance testing. In response, FEMA provided comments to the 
contractor, which resulted in a change in performance testing processes that 
ultimately is expected to improve system quality. 

IT Systems May Not Meet Expectations 

Given contractor delays and the many problems encountered, the MIP and 
related mapping tools may not fulfill user expectations. Specifically, users in 
some cases have acquired their own IT solutions and may not use the mapping 
tools that FEMA is providing. Delays in implementing the MIP have resulted 
in additional costs to maintain an interim system. In addition, FEMA has 
deferred its plans to integrate legacy systems with the MIP until the agency 
has the resources with which to do so. 

Mapping Tools May Not Be Used 

FEMA faces the risk that, despite spending approximately $4 million to 
develop tools for creating digital flood maps, mapping contractors and 
partners may choose not to use the tools. Because of contractor delays in 
providing the full suite of tools through the MIP, a number of mapping 
stakeholders have resorted to adopting other commercially available software 
to perform the same functions. For example, a mapping official we spoke 
with in California indicated that late release of the engineering tools had 
impeded his organization's ability to complete projects in a timely manner. 
Rather than continuing to wait for FEMA's tools, the contractor's 
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organization obtained its own tools. Further, another mapping contractor in 
Georgia had spent a lot of resources to independently create mapping tools 
and did not plan to use those provided by FEMA, especially since FEMA's 
tools will use an older version of geographic information system software than 
the contractor uses. FEMA officials fear that it may take months before they 
can determine the extent to which stakeholders will use their tools and 
whether the agency's investment to develop the tools is cost-effective. 

A contractor official stated that use of FEMA's tools is critical to the success 
of the map modernization program because these tools will effectively 
produce accurate flood maps, at lower cost, and in a standard manner. This 
official stated that the agency consequently must place priority on instituting 
incentives and highlighting the benefits of mapping stakeholders using the 
FEMA tools rather than their own IT solutions. One benefit to using FEMA's 
tools is that they will save money, as the stakeholders will not be responsible 
for purchasing and maintaining mapping tools on their own. Another 
potential benefit is the reduced requirement for mapping contractors and 
partners to manually enter mapping data into the MIP once the tools are fully 
integrated. 

Continued Reliance on Interim System 

Delays in MIP deployment also have resulted in increased costs to continue to 
operate an interim system for tracking mapping studies. Previously, FEMA 
and its mapping partners used the web-based Monitoring Information on 
Contracted Studies (MICS) system to record and track individual map 
modernization project lifecycles nationwide. In January 2005, the contractor 
created an interim system — commonly known as "son of MICS" — to prepare 
the data in MICS for transfer to the MIP. The transition process was only 
expected to take four to six weeks, during which time only the contractor's 
regional management center staff would be able to enter and update mapping 
information. At that point, FEMA expected the data transfer to be completed, 
and the MIP to be operational, by the end of February 2005. 

However, due to delays in implementing the MIP, regional management 
center staff are using the intermediary system much longer than originally 
intended to track the status of ongoing mapping studies. MIP release 2.2 is 
not expected to be fully operational until July 2005. Because mapping 
partners do not have electronic access, FEMA must continue to pay a 
contractor to update project tracking information in the intermediary system. 
Specifically, the contractor must spend approximately $150,000 per month to 
keep the management information up-to-date until release 2.2 of the MIP is 
implemented. If the MIP had been ready in September 2004 as originally 
scheduled, FEMA would not have had to pay these ongoing costs for activities 
that the agency or its mapping partners might have been done for free. 
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Legacy Systems Not Incorporated As Planned 



Due to MIP cost overruns, key legacy systems which were to be transitioned 
to the new mapping platform, will only be hyperlinked instead. Specifically, 
the contractor' s performance work statement for the map modernization 
program indicated that several legacy systems, including FEMA's 
Management Information System, MICS, and the Mapping Needs Update 
Support System, would be transferred to the MIP for easier access and use by 
mapping partners. Additionally, contractors stated in several technical 
documents and e-mails that they also would include the FEMA Levee 
Inventory System on the MIP. 

However, due to a lack of time and money to reengineer the mapping needs 
assessment processes to meet the MIP development deadline, FEMA agreed 
that the contractor should only link several systems to the MIP instead of 
integrating their entire databases with the new platform. Specifically, the 
Mapping Needs Update Support System — a multi-functional, interactive web- 
based software application — will not be fully included on the MIP and 
therefore will not provide the functionality that stakeholders originally 
expected. The system stores flood hazard mapping requirements and 
generates ranking reports for communities based on the benefits and costs to 
update maps. FEMA uses the system as a tool to prioritize communities for 
updates to their National Flood Insurance Program's Flood Insurance Rate 
Maps based on state and regional needs. Because processes to assess 
community needs are changing based on the MHIP and scoping activities and 
because new processes for identifying community needs have been deferred, 
FEMA officials reasoned that it would not be a good use of resources to 
integrate Mapping Needs Update Support System data into the MIP at this 
time and the hyperlink is a better option. FEMA officials anticipate that the 
new processes should be completed this fiscal year. 

Similarly, because of a lack of time and resources, the map modernization 
contractor has decided not to integrate the FEMA Levee Inventory System 
with the MIP. Nonetheless, because excess capacity exists on the MIP 
platform, the contractor has agreed to host the FEMA Levee Inventory System 
on the web site. 

Recommendations 

We recommend that the Acting Director, Mitigation Division: 

1 . Review and revise the Multi- Year Flood Hazard Identification Plan to 
improve sequencing and funding for mapping studies in high-risk areas to 
meet common goals and needs of the National Flood Insurance Program 
and local floodplain managers; and, to provide end-state objectives and a 
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long-term strategy for producing and maintaining accurate flood maps 
nationwide. 



2. Develop guidelines to help ensure compliance with FEMA's minimum 
standard for producing accurate and reliable flood insurance rate maps. 

3. Improve oversight of contractor program management by (a) providing 
guidance, additional training, and improved metrics for monitoring 
contractor performance; (b) documenting and ensuring compliance with 
procedures for reviewing and approving the contractor's IT procurements; 
and, (c) providing the resources needed to conduct the quality assessments 
and independent quality assurance reviews for monitoring contractor 
deliverables and services. 

4. Direct that the map modernization program office work with the DHS 
Geospatial Management Office to develop information sharing agreements 
with other federal agencies and programs to reduce redundancies in the 
costs and resources used to carry out map modernization activities. 

5. Finalize and implement the coordination policy for data sharing — and 
develop and implement a means to disseminate available federal agency 
mapping data to stakeholders — thereby capitalizing on available resources 
for meeting program objectives. 

6. Clearly communicate to stakeholders the objectives, benefits, and 
alternative approaches to converting from the former National Geodetic 
Vertical Datum of 1929 to the official North American Vertical Datum of 
1988 to help ensure accuracy of modernized maps. 

7. Improve FEMA regional office outreach and communication to 
stakeholders to help ensure their commitment and participation in efforts 
to modernize flood maps. 

8. Re-evaluate the Mapping Information Platform system development 
approach and make changes to ensure that (a) user and technical 
requirements are fully identified, approved, and addressed to support 
systems development; (b) contractor expectations are clearly defined; and, 
(c) standard methodologies for system development are followed to help 
manage costs, schedules, and results. 

Management Comments and OIG Evaluation 

We obtained written comments on a draft of this report from the Acting 
Director, Mitigation Division. We have included a copy of the comments in 
their entirety at Appendix B. 

Challenges in FEMA's Flood Map Modernization Program 
Page 39 



In the comments, the Acting Director concurred with all of the findings and 
recommendations in our report. The Acting Director said that the 
observations in our report are valuable to FEMA's ongoing improvement 
efforts and that the recommendations are generally consistent with the 
agency's current plans. The Acting Director said that FEMA, in fact, had 
made considerable progress over the previous few months with the flood map 
modernization program. In attachments to these general comments, and in 
response to each of our report recommendations, the Acting Director 
summarized progress that FEMA has made in flood map modernization and 
outlined plans for future improvements. We believe that such efforts are good 
steps toward addressing the various issues we raised in our report and look 
forward to learning more about continued progress and improvements in the 
future. 

Specifically, in response to Recommendation 1, the Acting Director outlined 
several actions that FEMA has taken, or will take, to review and revise the 
MHIP to meet stakeholder needs better. For example, the Acting Director 
said that updates to the MHIP in FY 2006 to 2008 will include improvements 
in projected flood map modernization sequencing and, as available, funding 
for high-risk areas. This updated information will be accompanied by a 
description of differences between mapping and funding needs addressed in 
the MHIP and those needs identified in state and regional business plans. 
Further, the Acting Director stated that program managers are working to 
define end-state objectives of flood map modernization and develop a long- 
term strategy for producing and maintaining accurate flood maps nationwide. 
To ensure their engagement and commitment, stakeholder comments on the 
MHIP and its subsequent revisions will be incorporated in the long-term 
planning efforts. 

In response to Recommendation 2, the Acting Director indicated that FEMA 
is holding a series of meetings with representatives of state and local partners 
to obtain input and agreement on how best to implement and ensure 
compliance with the agency's quality standards. The agency also had 
published within the previous month guidance and protocols for conducting 
flood studies in the Atlantic and Gulf Coast regions. 

The Acting Director outlined various efforts to address Recommendation 3, 
regarding the need for improved oversight of contractor program 
management. Specifically, the Acting Director stated that FEMA is 
strengthening its use of the Earned Value Management System for reporting 
and tracking performance in carrying out contract task orders and flood 
studies. The Acting Director indicated that such strengthened performance 
management would be accomplished through improved metrics, training, 
integrated baseline reviews, and formal monthly evaluations of flood map 
modernization program status. The Acting Director said that FEMA will 
provide continued enforcement of the IT procurement policy in place to 
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ensure pre-approval of any IT purchases over a set dollar amount. Further, 
FEMA continues to use its quality control contractor to ensure the quality of 
deliverables and affirms that actions will be taken to address 
recommendations made by that contractor. 

The Acting Director discussed in his comments FEMA's commitment to 
addressing our report Recommendations 4 to 7, regarding communication, 
outreach, and partnership on map modernization activities. Specifically, the 
Acting Director stated that FEMA will continue to work closely with the DHS 
Geospatial Management Office to develop and formalize data sharing 
agreements with other organizations, such as the National Geodetic Survey, 
the U.S. Army Corps of Engineers, and the U.S. Geological Survey, to reduce 
overlaps and maximize efficiencies. FEMA recently finalized, and is working 
to implement, its data coordination policy to help govern the dissemination of 
federal mapping data to stakeholders. FEMA recognizes the importance of 
developing improved guidance and communicating to stakeholders on the 
need to use the official elevation data standard to help ensure the accuracy of 
modernized flood maps. Further, although FEMA already has taken steps to 
improve regional outreach, the agency also will provide guidance to its 
regional offices, emphasizing increased coordination with state map 
modernization stakeholders to ensure their commitment and participation. 

Finally, in response to Recommendation 8 on re-evaluating the mapping 
platform development approach, the Acting Director stated that requirements 
analysis meetings and pilot tests help ensure that system requirements are 
clearly identified, documented, and approved. Daily and weekly meetings 
also are held to review system development status, identify problems, and 
remove obstacles to program success. FEMA will use independent 
verification and validation processes to enforce use of a proven, standard 
system development methodology. The agency will link the implementation 
of process and product improvement recommendations resulting from these 
independent verification and validation processes to future metrics for IT 
project management. 
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As background for our review, we researched U.S. laws, federal guidance, and 
DHS directives relating to floodplain management, mapping, and the National 
Flood Insurance Program. We obtained documentation through internet 
searches relating to map modernization, mapping communities, IT 
requirements and FEMA guidelines and specifications for mapping partners. 
Additionally, we reviewed FEMA OIG and GAO reports to learn more about 
their findings and recommendations related to map modernization. 

To accomplish our review objectives, we first met with FEMA representatives 
to learn about their roles, responsibilities, and activities relating to the Multi- 
Hazard Flood Map Modernization Program. We conducted individual 
interviews with FEMA's Chief Information Officer and the Geospatial 
Information Office to identify management directives that apply to map 
modernization's IT infrastructure and to evaluate FEMA's geospatial 
coordination and subsequent reporting on those efforts. In addition, we held 
meetings with FEMA's Risk Identification Branch Chief and map 
modernization staff responsible for contract or oversight; communications and 
outreach; IT development, management, and acceptance; and engineering 
functions. These meetings helped us gauge the effectiveness of FEMA's 
contract management, coordination, and IT management activities as well as 
broaden our understanding of the map modernization program in general. 

We interviewed staff of the primary contractor selected to support map 
modernization efforts, to learn about progress, challenges, and successes they 
encountered in managing the program. The contractor' s team leader and staff 
members assisting in the program told us about their experiences with FEMA 
and gave us periodic status updates on the program. Also, we interviewed 
subcontractors, indefinite delivery/indefinite quantity contractors, and 
individual cooperative technical partners-sanctioned contractors to determine 
the quality of FEMA's management of the program, communication and 
coordination, with stakeholders, and use of IT to support mapping efforts. 

We met with a number of stakeholders representing the Map Modernization 
Coalition that supported funding for map modernization. These stakeholders 
included the Association of State Floodplain Managers and the American 
Congress of Surveyors and Mappers. The goal of these meetings was to 
determine stakeholder satisfaction with the program's status, learn about 
communication and outreach activities, and evaluate management of the IT 
infrastructure. 

We held interviews with agencies with which FEMA coordinated and 
collaborated on map modernization issues. These included the U.S. 
Geological Survey, the National Oceanic and Atmospheric Administration's 
National Geodetic Survey, the U.S. Army Corps of Engineers and the U.S. 
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Census. Through the course of these meetings, we obtained information 
relating to FEMA's participation in the Federal Geographic Data Committee, 
the National Digital Orthophoto Program, and the National Digital Elevation 
Program, as well as its support to Geospatial One-Stop. 

During the course of our review, we conducted site visits to three FEMA 
Regions across the U.S., selected based on mapping activity and suggestions 
by FEMA officials. We visited FEMA Regions IV, V, and IX, all of which 
had large numbers of mapping studies, varying levels of cooperative technical 
partner participation, and unique map needs and characteristics. In each 
location, we visited with representatives of the FEMA regional offices, 
regional management centers, mapping partner organizations, and indefinite 
delivery /indefinite quantity contractors. Subcontractors interviewed provided 
a broad understanding of stakeholder needs, communication and IT 
requirements, as well as progress made to date. These interviews assisted in 
our ongoing research of FEMA's overall management of the Multi-Hazard 
Flood Map Modernization Program, as well as the agency's coordination and 
communication with all levels of government, mapping partners, and other 
stakeholders. In addition, we evaluated the agency's multi-million dollar MIP 
system, its requirements, specifications, capabilities, and overall usefulness. 

We attended FEMA's 2005 Map Modernization Conference in Kansas City, 
Missouri. The conference, an annual event at which all Map Modernization 
Program Staff convene, focused heavily on performance-based management 
as well as programmatic implementation issues. During the conference, we 
attended formal working sessions ranging in topics from cooperating technical 
partner development to discussions concerning the next release of the MIP. 
Our participation in these events broadened our understanding of the program 
and FEMA's management, coordination, and use of IT infrastructure to meet 
program goals. 

We conducted our review from August 2004 through April 2005 at FEMA 
headquarters and related government and industry organizations in the 
Washington, D.C. metropolitan area, as well as in FEMA Region V (Illinois 
and Wisconsin), Region IX (Arizona and California), Region IV (Florida, 
Georgia, Alabama, and North Carolina), and at FEMA's 2005 Map 
Modernization Conference in Kansas City, Missouri. We performed our work 
according to generally accepted government auditing standards. The principal 
OIG points of contact for this audit are Frank Deffer, Assistant Inspector 
General, Information Technology Audits and Sondra McCauley, Director, 
Information Management Division. Other major contributors are listed in 
Appendix C. 
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U.S. Department of Homeland Security 
Washington, D.C. 20472 



m FEMA 

SEP 6 20» 



MEMORANDUM FOR; 



FROM: 



SUBJECT; 



General Comments 

The FEMA Mitigation Division thanks you for the opportunity to comment on the subject 
report. The draft report contains significant observations and recommendations about the 
Flood Map Modernization Program. FEMA concurs with the findings and 
recommendations listed in the report. The observations are valuable to our improvement 
efforts, and the recommendations are generally consistent with our current plans. In fact, 
FEMA has made considerable progress over the course of the past few months with the 
Flood Map Modernization Program, and our response will summarize some of the 
progress FEMA has made as well as our plans for future improvements. 

Over the course of the past year, FEMA's Flood Map Modernization Program has 
achieved much in moving toward the objectives of this initiative. Because of the Flood 
Map Modernization Program, citizens will be able to make better decisions about the use 
of their property, both inside and outside the flood prone areas identified on the maps. 
Communities will be provided with a more comprehensive approach to disaster 
mitigation planning, economic development, and emergency response. Communities also 
will be able to manage flood risks, water resources, land use, and other responsibilities 
more effectively. Finally, through Flood Map Modernization, communities will be 
empowered to update maps and data as risks change. 

Through oversight of this initiative, FEMA is undertaking groundbreaking steps in 
designing and implementing performance-based processes in the mapping environment. 
This oversight is necessary to achieve the mission of this program: to protect lives and 
prevent loss of property from flood hazards. 

1 



Frank Deffer 

Assistant Inspector General, Information Technology 
David Maurstad 

Acting Director, Mitigation Division 

Response by the FEMA Mitigation Division to the Draft 
Audit Report - Challenges in FEMA '$ Flood Map 
Modernization Program (OIG-05-XX) 
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The OIG report presented the results of its audit on the following four major topics: 

• Limitations of the Flood Map Modernization Program (Map Mod) plan as 
documented in the Multi-Year Flood Hazard Identification Plan (MHIP); 

• Improvements needed in program management approach; 

• Communication and coordination activities with Federal, State, and local entities; 
and 

• Information technology (IT) issues, with particular emphasis on the FEMA 
Mapping Information Platform (MIP). 

Attachment A presents FEMA's progress on these four major topics. The OIG report also 
provides eight specific recommendations for FEMA actions to improve Map Mod. 
Attachment B presents specific responses to the eight OIG recommendations. 
Attachment C lists the acronyms referenced in the response. 

In summary, with the support of FEMA senior management, the Mitigation Division is 
committed to addressing the issues identified in this report by planning and implementing 
activities within the available funding, timing, and resources. 

Additional questions regarding this response may be directed to Michael F. Howard in 
my office at (202) 646-4070 or at MLchaelF.Howard@dhs.gov . 

Attachments 
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Attachment A 

Summary of Progress on Four Major Topics in the IG Report 
Major Topic #1: Map Mod Plan Limitations 

Mapping Plans and Priorities, The Multi-Year Flood Hazard Identification Plan 
(MHTP) was prepared in response to recommendations given by GAO Review in March 
2004. FEMA prepared a draft document in June 2004. Version 1.0 was published in 
November 2004; Version 1.5 was released in June 2005. 

Funding and User Requirements. The OTG report states the MHIP is a "comprehensive 
plan that outlines priorities, funding, and standards for carrying out mapping studies." 
However, the report also states that MHIP "is hindered by budget limitations and does 
not adequately address user or funding requirements" The FY 2004-2008 budget used in 
MHTP reflects funding received and anticipated by FEMA from the President and 
Congress. FEMA recognizes that this level of funding does not meet all the needs of our 
State and local mapping partners; however, it is important to note that FEMA's role in 
flood map modernization focuses on essential flood mapping requirements and may not 
include all State flood map needs . FEMA's role in flood map modernization must be 
complemented by State and local mapping partners. FEMA is currently evaluating the 
level of funding required for flood map maintenance. 

FEMA agrees that clear communication among community, State, and regional partners 
and other mapping stakeholders is a critical factor for the success of Flood Map 
Modernization. FEMA reviewed all comments from stakeholders on the November 
2004 version of MHIP and the most recent June 2005 version of MHIP. Continuous and 
consistent lines of communication at multiple levels drive the development and update 
process, and they are resulting in better- informed stakeholders and users and better 
mitigation planning. 

Standards and Guidelines. The OIG report states that "stakeholders commended 
FEMA for establishing the new quality standard." However, the report also states that 
stakeholders "expressed concern that MHIP does not provide guidelines for evaluating 
modernized maps to ensure that the standard is met." FEMA publishes Guidelines and 
Specifications for Flood Hazard Mapping Partners. The document defines technical 
requirements, product specifications for flood hazard maps and related NFIP products. 

Several steps have been taken towards guidance development and implementation. Some 
of these steps are summarized below: 

• FEMA Headquarters and Regions have been discussing the implementation 
process throughout the winter and spring. 

• A procedure memorandum has been issued requiring all FY05 studies to comply 
with Section 7 of the MHTP. 

• Interna] draft guidance has been written. 
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• FEMA hosted an Industry Day discussion on June 17, 2005, with its mapping 
partners to help resolve Section 7 implementation issues, 

» Procedures for implementing the floodplain boundary standard were discussed 
during a meeting with State and local government staff on August 3 and 4, 2005. 

Long-Term Strategy and Map Maintenance. The OIG report states that "MHIP does 
not present a long-term strategy for map modernization" and that "MHIP does not 
adequately address costs for map maintenance and re-study." MHIP is a roadmap that 
defines, for the first time, how FEMA will produce updated digital flood hazard data for 
the Nation. The overall strategy, presented in Section 1 of the FY05-FY09 MHIP 
(Version 1.5, June 2005), states that Flood Map Modernization will result in safer 
communities by providing more reliable, readily available, and easier-to-use flood maps. 
The following is a brief summary from MHIP of the long-term actions: 

. Networking the Nation using Internet technology to provide access to general 
fiood hazard, risk, and mitigation information. 

. Maximizing the use of local, State, and Federal resources, and transferring 
ownership and use of maps and data to the States and localities by building and 
maintaining effective partnerships with community, State, and regional entities 
before and during the development of maps and data. 

• Reducing the processing time and cost for map updates, which includes map 
maintenance and re-study, and increasing accountability for spending by 
implementing results -oriented systems and standards that will facilitate the rapid 
exchange of data between mapping partners, stakeholders, FEMA staff, FEMA 
contractors, and other users. 

• Communicating with mapping partners, stakeholders, and users effectively, 
consistently, and continually to maximize understanding of flood hazards and the 
risks that these hazards pose to life and property. 

• Continuing to improve the quality and accuracy of national flood hazard data by 
developing GIS-based products with reliable technologies that meet enhanced 
technical standards such as the floodplain boundary mapping standard defined in 
section 7 of the FY04-FY0B MHIP (Version 1.0), dated November 2004. 

The long-term map maintenance issues are being addressed by capturing the data in 
digital form and by maintaining the data in easily accessible systems. FEMA has 
developed and published data capture standards and guidelines (Appendix N of FEMA's 
Guidelines and Specifications for Flood Hazard Mapping Partners). These guidelines 
will also facilitate and reduce re -study costs. 

Major Topic #2: Program Management Improvements 

Management oversight reflects the growing sophistication in understanding the properties 
Of the earned value management system (EVMS). The EVMS tables and statistics are 
standard features in the monthly status reports. The monthly joint program reviews (JPR) 
between FEMA and MOD use these management reports as the basis for tracking 
progress, issues of concerns, or areas at risk. Over the past year, the program staff has 
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changed from knowing little about EVMS to using the terms, concepts, and analyses to 
manage their areas of responsibility and to discuss Map Mod's progress in a standardized 
and easily understood manner. 

The following objectives have been set for the JPR meetings: 

• Understand program performance to date 

• Analyze current earned value data 

• Align review with core business processes 

• Review process improvements in response to Independent Verification and 
Validation (IV &V) or external recommendations 

Major Topic #3: Communication and Coordination Issues 

Guidance on Communication and Coordination. 

The overarching goal of outreach is to create a climate of understanding and 
ownership of the mapping process at the State and local levels. Well -planned 
outreach activities reduce political stress, confrontation in the media, and public 
controversy. 

Specific mapping-project-related outreach goals include the following: 

• Establishing two-way communication to inform and obtain feedback from 
stakeholders; 

• Ensuring compliance with due process requirements; 

• Interacting with technical representatives to ensure production of accurate and up- 
to-date maps; 

• Identifying and addressing the needs of all affected stakeholders; 

• Enhancing ownership by communities; 

• Tracking, monitoring, and evaluating outreach activities and adjusting efforts 
according to ongoing feedback and evolving project needs. 

Geospatial Data Sharing Partnerships and Guidance. FEMA also recognizes the 
importance of coordinating with its Federal, State, and local partners to avoid duplication 
of data acquisition and to encourage the sharing of data. Fostering collaboration with 
other Federal agency partners (e.g., U.S. Army Corps of Engineers (US ACE), U.S. 
Geological Survey (USGS), National Oceanic and Atmospheric Administration's 
National Geodetic Survey (NOAA/NGS), Natural Resources Conservation Service) and 
with State, regional, and local agency partners through the Cooperating Technical Partner 
(CTP) program will help improve and maintain the quality and reliability of flood maps 
and other flood hazard data. 

During the last 6 months of FY 2005, FEMA has been very active in the area of 
geospatial data coordination, issuing a formal policy on geospatial data coordination, 
implementing this policy across the Regional offices and establishing a geospatial data 
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coordination and standardization management team to support the implementation of the 
policy. The goals of the team are: 

• Integrating Map Mod data coordination and standardization activities with day-to- 
day planning and project management activities; 

• Facilitating successful local relationships being built among local, State and 
Federal personnel; 

• Implementing processes that will help FEMA document productive coordination 
at the national level and report on geospatial investments and data holdings. 

FEMA is supporting the NSDI and Geospatial One-Stop (GOS) as prescribed by Office 
of Management and Budget Circular A- 16 and Executive Orders 12906 and 13286. 
FEMA is implementing a geospatial metadata management system that will automatically 
provide the NSDI and GOS with information about geospatial datasets used in Flood 
Map Modernization. 



Major Topic #4: Information Technology Issues 

Coordinated IT Approach. Despite the challenges described in the report, FEMA has 
released a comprehensive suite of engineering and map production tools and workflow 
functionality on the Mapping Information Portal (MIP). FEMA and NSP subject matter 
experts (SMEs) collaborated in frequent software test status meetings to prioritize 
problems found after testing the system from a business user perspective. These efforts 
contributed significantly to the delivery and approval of Releases 2,1 and 2.2 (currently, 
conditional approval only for Release 2.2); the SMEs are actively using their in-depth 
MIP understanding in training and deployment. 

IT Systems Meet Expectations. Actual use of the engineering and map production tools 
is growing. On August 9, there were 1 14 logons to DF1RM tools versus an average of 
30 logons per day in the period immediately following release of the tools. Feedback 
from workflow users has been positive during the first wave of the deployment in June, 
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Attachment B 



Mitigation Division's Response to Recommendations in the IG Report 

The FEMA Mitigation Division makes the following response to each recommendation: 

Regarding Recommendation #1 - Reviewing and revising the Multi-Year Flood 
Hazard Identification Plan to improve sequencing and funding for mapping studies 
in high-risk areas to meet common goals and needs of the National Flood Insurance 
Program and local flood plain managers; and, to provide end-state objectives and a 
long-term strategy for producing and maintaining accurate flood maps nationwide. 

The purpose of FEMA's Multi-Year Flood Hazard Identification Plan (MHIP) is 
to share with our mapping stakeholders the planned schedule and anticipated 
funding for providing reliable digital flood hazard data and maps for areas at 
flood risk to support the National Flood Insurance Program. State, local, and 
regional entities provide input to the MHIP through the State and Regional flood 
mapping business planning process. The MHIP provides a national roll -up of the 
State and regional information and county-specific projected dates for delivery of 
flood hazard maps, to aid planning efforts of FEMA's flood hazard mapping 
partners. FEMA's role in flood map modernization focuses on essential 
requirements and may not include all State flood map needs. FEMA's role in 
flood map modernization must be complemented by State and local mapping 
partners. 

FEMA released the first version of the MHIP (Version 1.0) in November 2004 
with a strategy to update the plan twice yearly to reflect progress on delivering 
flood data and maps, for use and adoption by communities, and to reflect changes 
in cost and schedule for planned flood map updates. The MHIP web page on 
FEMA's Flood Hazard Mapping Website provides opportunity to comment on the 
Plan. Flood mapping stakeholder participation in development and feedback on 
the MHIP is essential to the success of Flood Map Modernization. The first 
update to the MHIP (Version 1.5), released in June 2005, contains information 
addressing the most frequently expressed comments on the first MI-HP, 
sequencing updates, and updates to Hood hazard map schedule and cost. 

FEMA will improve the flood- map update sequencing and, as available, funding 
in high-risk areas in Multi-Year Flood Hazard Identification Plan (MHIP) 
updates. Since the issuance of the IG report, FEMA has met with 
representatives of State and local flood hazard mapping partners in developing a 
coordinated strategy to address issues of mapping quality standards, strategy to 
address mapping issues with levees, , and for resolving issues related to defining 
flood risks that impact sequencing and level of mapping detail. Additionally, 
agreement was achieved in discussing end state objectives for producing and 
maintaining accurate flood maps. Policy statements on mapping quality standards 



Page 50 



Appendix B 

Management Response To Draft Report 



and mapping levees have been issued. Upcoming planned actions include updates 
to the sequencing of studies in the MHIP to reflect coordinated strategies for 
identifying flood hazards for areas at risk and funding thresholds for flood 
mapping for studies planned for the FY06-FY08 timeframe. These updates will 
include a description of differences between those needs being addressed in the 
MHIP and those needs identified in the State and Regional business plans. Over 
the next six months, FEMA, in coordination with State and local government 
mapping partners is working to clarify the end-state objectives and develop a 
long-term strategy for producing and maintaining accurate flood maps 
nationwide. This includes applying benefits of digital flood mapping technology 
implemented through FY08 to addressing long-term flood data update needs. The 
comments received on the MHIP inform and refine that process, and subsequent 
MHIP updates will reflect the results of this long-term planning and facilitate 
continued stakeholder engagement and buy-in. 

Regarding Recommendation #2 - Developing guidelines to help ensure compliance 
with FEMA's minimum standard for producing accurate and reliable flood 
insurance rate maps. 

In response to a March 2004 GAO report, FEMA included a floodplain mapping 
standard, aligned with the level of flood risk, in the November 2004 MHIP 
Version 1 .0. That document served as a means of making the draft standard 
available for stakeholder comment. FEMA's flood mapping standards are 
defined in FEMA's Guidelines and Specifications far Flood Hazard Mapping 
Partners. The comments received on the draft mapping standard in MHTP 
Version 1 .0 are being considered as FEMA incorporates the refined mapping 
standard in our Guidelines and Specifications. 

Also, FEMA held and will continue to hold meetings with representatives of our 
State and local partners, including one held on August 3 and 4, 2005. During this 
meeting, quality guidelines and compliance procedures were agreed upon in order 
to ensure compliance with flood zone boundary mapping standards. Through 
revised State business planning process, States will address how they intend to 
ensure compliance with mapping standards and address specific issues relating to 
high risk areas involving levees. Recently published guidance within the past 30 
days includes "Procedure Memorandum No. 37 - Protocols for Atlantic and Gulf 
Coast Coastal Flood Insurance Studies in FY05 " and "Procedure Memorandum 
No. 34, Interim Guidance for Studies Including Levees." 



Regarding Recommendation #3 - Improving oversight of contractor program 
management by (a) providing guidance, additional training, and improved metrics 
for monitoring contractor performance; (b) documenting and ensuring compliance 
with procedures for reviewing and approving the contractor's IT procurements; 
and, (c) providing the resources needed to conduct the quality assessments and 
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independent quality assurance reviews for monitoring contractor deliverables and 
services. 

As the map modernization initiative makes progress, the governance process will 
be changed to manage better the performance of the contract and have greater 
program oversight. With the improved governance structure, there will be three 
areas of oversight and control - the portfolio, the program, and the process/project 
levels. 

For all three of these levels, FEMA will strengthen its use of Earned Value 
Management System (EVMS) for reporting and tracking of major task orders and 
flood studies. Special emphasis will be placed on defined, measurable, and 
mutually agreed-upon metrics and products. Part of strengthening EVMS will be 
by giving training and refresher courses to program staff on EVMS methodology, 
conducting integrated baseline reviews, and instituting formal monthly 
evaluations of the status of Flood Map Modernization current operations. 
Program managers and staff received extensive training in the techniques and 
analysis of earned value management data during the first year of the program. 

To strengthen FEMA management and oversight of this initiative and comply 
with DHS MD 0782, "Program Manager Certification," the program managers 
will achieve appropriate program management training to qualify at Level II in 
FY 2006. Over the past year, program staff received extensive management 
training to achieve Level I proficiency or better. 

In addition, FEMA will provide continued enforcement of the information 
technology (IT) procurement policy recently put in place. That policy requires 
that before procurement of any IT item over $5000, the contractor must obtain 
pre-approval by the Applications and Analysis Section. FEMA will also review 
IT purchases at weekly IT status meetings and document all approvals and 
disapprovals in writing. 

Since the review by the Inspector General, FEMA's 1V&V contractor has become 
fully engaged in the program delivery and the review of deliverables from the 
National Service Provider. For important deliverables, the 1V& V contractor 
reviews deliverables and provides and assessment to FEMA. These assessments 
are also provided to the National Service Provider. Future National Service 
Provider task orders will have metrics linked directly to quality assessments 
performed by the IV&V contractor. 

At the monthly Joint Program Review meetings, each business process area 
includes a discussion of actions taken to address recommendations from the 
IV&V contractor. 

Regarding Recommendation #4 - Directing that the map modernization program 
office work with the DHS Geospatial Management Office to develop information 
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sharing agreements with other federal agencies and programs to reduce 
redundancies in the costs and resources used to carry out map modernization 
activities. 

Map Modernization will continue to work closely with the DHS Geospatial 
Management Office (GMO) to develop and formalize data sharing agreements 
with other agencies to reduce overlaps and maximize efficiencies. The DHS 
GMO is finalizing a formal agreement with USGS that supports improved 
coordination between DHS and USGS. FEMA, along with the head of the DHS 
GMO, met with the USGS GIO on August 8 th to continue to build on these efforts 
and to develop a supporting service level agreement specific to coordination 
between FEMA and USGS, Map Modernization will also continue to pursue 
formal agreements with NO A A/ NGS, USAGE and other key partners. 

Regarding Recommendation #5 - Directing the contractor to finalize and implement 
the coordination policy for data sharing — and develop and implement a means to 
disseminate available federal agency mapping data to stakeholders — thereby 
capitalizing on available resources for meeting program objectives. 

FEMA recognizes the importance of coordinating with its Federal, State, and 
local partners to avoid duplication of data acquisition and to encourage the 
sharing of data. On August 23 rd , FEMA issued its "Geospatial Data Goordination 
Policy." Regional offices are working to implement the data coordination policy. 

The policy, when coupled with data coordination tools implemented in the 
Mapping Information Platform, is the flood program's means to disseminate 
available federal mapping data to stakeholders. The data coordination tools 
include the hazard catalog; the National Digital Elevation Program and the 
National Digital Orthophoto Program project tracking system; and the metadata 
profiles for imagery, elevation, and flood data. These geospatial data 
coordination tools also provide information about Flood Map Modernization data 
to the Geospatial One-Stop. FEMA will work with its Regions, partners, and 
contractors to ensure that these tools are used in accordance with the data policy 
and implementation guidance. FEMA will also establish an audit process to 
ensure that the data policy is being adhered to. 

Regarding Recommendation #6 - Clearly communicating to stakeholders the 
objectives, benefits, and alternative approaches to converting from the former 
National Geodetic Vertical Datum of 1929 to the official North American Vertical 
Datum of 1988 to help ensure accuracy of modernized maps. 

FEMA agrees that it is vital to communicate effectively with stakeholders about 
the importance of using NAVD 88 on the new flood maps. The Engineering and 
Mapping Integrated Program Team (IPT) will develop and communicate 
improved guidance on this issue, FEMA will also improve coordination 
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processes with the National Geodetic Survey to enlist their support in 
communicating the importance of this issue to stakeholders. 

FEMA guidance for converting elevation data from the National Geodetic 
Vertical Datum of 1929 (NGVD29) to the North American Vertical Datum of 
1988 (NAVD88) is provided in Appendix B of the previously cited Guidelines 
and Specifications. In the coming months, the FEMA Map Mod program office 
will develop and communicate improved guidance on the issue of converting 
elevation data from NGVD29 to NAVD88. At the same time, FEMA will review 
current projects and task orders to insure that NAVD88 is used where appropriate. 

Regarding Recommendation #7 ~ Improving FEMA regional office outreach and 
communication to stakeholders to help ensure their commitment and participation 
in efforts to modernize flood maps. 

FEMA has already taken steps to improve regional office outreach by conducting 
a review for each office of its staffing needs and effectiveness for outreach 
activities. Each of the 10 Regions has taken steps over the past year to increase 
communication with mapping partners, States and locals, and their Regional 
Management Center (RMC) contractor. Specifically, all 10 Regions hold 
mapping conferences at least once per year, and often 2-3 times annually. These 
conferences provide an opportunity for regional staff, mapping partners, local 
stakeholders, and the RMC to discuss best practices and common issues that 
require attention. While regional staffs regard personal contact as the best form of 
communication, the NSP has developed some very useful tools, including a brief 
summary of the program included with all preliminary maps. 

FEMA will provide its regional offices with guidance to emphasize increased 
coordination with cognizant State flood mapping officials to ensure state 
commitment and participation. This change will provide better management of 
prospective mapping resources and provide an opportunity to leverage outreach 
opportunities at die state level. In addition, this change will enable more efficient 
leveraging of State resources to meet both local and national needs through the 
Cooperating Technical Partners (CTPs). Current guidance documents are being 
refined to provide assistance based upon lessons learned. Resources are being 
redirected to encourage partner innovation and practices that will benefit flood 
map modernization through better business management and outreach. 

Regarding Recommendation #8 - We recommend that the Acting Director, 
Mitigation Division re-evaluate the Mapping Information Platform system 
development approach and make changes to ensure that (a) user and technical 
requirements are fully identified, approved, and addressed to support systems 
development; (b) contractor expectations are clearly defined; and, (c) standard 
methodologies for system development are followed to help manage the costs, 
schedule, and results. 
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FEMA has already taken steps to improve the Mapping Information Platform (MIP) 
requirements, clearly define contractor expectations, and help enforce the use of standard 
methodologies for system development. 

a) MIP Requirements Defined with MIP Phased Roll Out: FEMA has had 
several detailed requirements analysis meetings with FEMA and key stakeholders 
to ensure that all requirements are clearly identified, documented, and approved. 
FEMA officially approved the MIP Release 3 requirements on May 6, 2005 . 
Additionally, FEMA has piloted the MIP Release 2.2 system out to Regions 6, 8, 
and 10 (including their contractors and Cooperating Technical Partners), The 
feedback from the Regional roll-out sessions has been extremely positive. Those 
Regions are now actively using the MIP. From those pilot sessions, requirements 
and change requests were recorded and prioritized to clearly define future 
requirements. The remaining Regions will complete training through September 
and early October 2005. At that point the system will be fully rolled out and used 
by all Regions. 

b) Contractor Oversight Improved: Daily status meetings of the progress of the 
system are conducted. Additionally, weekly detailed status meetings are being 
conducted to ensure contractor expectations are clearly understood. At each 
weekly meeting, FEMA reviews the project using Earned Value Management 
System (EVMS) techniques tracking cost and schedule. FEMA views Cost (and 
Effort) Performance Index measures and Schedule Performance Index measures 
to alert us to problems with cost and schedule. When Cost/Effort or Schedule 
variance exceeds 5%, FEMA determines the cause of the variance by identifying 
the work packages in our detailed schedule that are causing the problems. Once 
issues are identified, FEMA takes action to remove obstacles. This has worked 
effectively to keep us on schedule and on budget to date with our Task Order 1 
Extension. To ensure that contractor expectations arc clearly defined for our next 
core Task Order, FEMA jointly worked with the Mapping on Demand (MOD) 
team to provide input into their Performance Work Statement, FEMA intends to 
continue open dialog through the contracting process to ensure that expectations 
are clearly documented and understood by both FEMA and the MOD team. 

c) Standard Methodology Used with IV &V: Systems Engineering Institute (SEI) 
Capability Maturity Model Integration (CMMI) Level 3 processes, a proven 
standard methodology for systems development, will be enforced for the IT 
sections of the program. FEMA is currently using our Independent Verification 
and Validation (IV&V) contractor to verify that those processes are being 
followed. Recommendations from our 1V&V contractor have been sent to the 
MOD team. Actions such as MOD's improvement of their IT Configuration 
Management process have resulted from IV&V recommendations. This review 
process will be continuous; in addition, FEMA is looking to tie future metrics to 
implementation of IV&V recommendations on quality of products and process 
improvements/compliance. As noted in (b), EVMS is used to help track budget 
and schedule and to identify areas of risk to prevent schedule and cost overruns. 
Additionally, the detailed scheduled used to track the system status and EVMS is 
mapped to the standard development methodology. Deviations from schedule or 
budget are immediately addressed; if no solution can be found to contain the 
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variance then the Joint Program Review will be briefed on the issues for 
resolution. 

Better controls over requirements, contractor expectations, and project 
management have been put in place over the last few months to help the MiP 
succeed. The benefits of these controls are starting to be realized (e.g., Release 3 
is currently on schedule and budget and improved processes have been put into 
place to improve Configuration Management System Testing per IV&V 
comments). 
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ACRONYMS 
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FfiiTirH Vfllnf Management Svstcm 


GMO 


Geospatial Management Office 


COS 


Geo-Spatial One-Stop 


1PT 


Integrated Program Team 


IT 


Information Technology 


IV&V 


Independent Verification & Validation 


JPR 


Joint Program Review 


MHIP 


Multi-Year Flood Hazard Identification Plan 


MIP 


Mapping Information Platform 


MOD 


Mapping on Demand, (consortium supporting Map Mod) 


NAVD88 


North American Vertical Datum 


NDEP 


National Digital Elevation Program 


NDOP 


National Digital Orthophoto Program 


NFIP 


National Flood Insurance Program 


NGS 


National Geodetic Survey 


NGVD29 


National Geodetic Vertical Datum 


NOAA 


National Oceanic and Atmospheric Administration 


NSDJ 


National Spatial Data Infrastmcture 


NSP 


National Service Provider 


RMC 


Regional Management Centers 


SME 


Subject Matter Experts 


USACE 


U.S. Army Corps of Engineers 


USGS 


U.S. Geological Survey 
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